Jul 15 21:06:34.264771 ip-10-0-128-44 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 21:06:35.953586 ip-10-0-128-44 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 21:06:35.953586 ip-10-0-128-44 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 21:06:35.953586 ip-10-0-128-44 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 21:06:35.953586 ip-10-0-128-44 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 21:06:35.953586 ip-10-0-128-44 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 21:06:35.955424 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.955303    2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 21:06:35.963508 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963487    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:06:35.963508 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963506    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:06:35.963508 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963510    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:06:35.963508 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963513    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963516    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963520    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963524    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963527    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963530    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963532    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963535    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963537    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963540    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963543    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963545    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963548    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963550    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963554    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963556    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963559    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963562    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963565    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963570    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:06:35.963658 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963573    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963578    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963583    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963585    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963588    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963591    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963593    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963596    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963598    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963601    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963603    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963606    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963608    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963611    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963614    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963616    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963619    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963622    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963624    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:06:35.964124 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963627    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963630    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963632    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963636    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963639    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963642    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963645    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963647    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963650    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963652    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963655    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963657    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963660    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963664    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963667    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963670    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963672    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963675    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963677    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963680    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:06:35.964553 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963682    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963685    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963688    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963690    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963693    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963695    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963698    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963700    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963703    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963706    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963708    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963711    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963714    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963717    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963719    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963722    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963725    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963727    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963730    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963733    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:06:35.965057 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963736    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963739    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963741    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963744    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963747    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963749    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963752    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963754    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963757    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963759    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963763    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963767    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963770    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963773    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963776    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963778    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.963781    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964478    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964484    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:06:35.965534 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964488    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964491    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964494    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964496    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964499    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964503    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964506    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964508    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964511    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964513    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964516    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964520    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964524    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964527    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964531    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964534    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964537    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964539    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964542    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964545    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:06:35.965996 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964547    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964550    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964552    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964555    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964558    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964561    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964564    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964566    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964569    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964573    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964577    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964580    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964583    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964586    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964589    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964592    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964595    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964598    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964600    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964603    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:06:35.966515 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964606    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964608    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964611    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964614    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964617    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964620    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964622    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964625    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964628    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964631    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964633    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964636    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964638    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964641    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964643    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964645    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964648    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964650    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964652    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:06:35.967014 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964655    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964658    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964660    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964663    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964666    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964668    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964671    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964673    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964676    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964679    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964681    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964683    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964686    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964688    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964691    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964694    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964696    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964698    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964701    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964705    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:06:35.967492 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964707    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964710    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964713    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964717    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964720    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964723    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964725    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964727    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964730    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964732    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964735    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964737    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964740    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964743    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964745    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964748    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964751    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.964753    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965168    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965174    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:06:35.967980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965177    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965179    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965182    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965185    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965188    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965190    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965193    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965195    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965198    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965200    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965203    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965205    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965209    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965213    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965216    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965220    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965224    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965227    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965230    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:06:35.968488 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965233    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965235    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965238    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965241    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965244    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965246    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965249    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965252    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965255    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965258    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965260    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965263    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965265    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965267    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965270    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965272    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965275    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965277    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965280    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965282    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:06:35.968931 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965285    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965288    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965291    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965293    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965296    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965298    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965300    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965303    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965306    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965309    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965311    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965314    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965316    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965319    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965322    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965324    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965326    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965330    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965333    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:06:35.969423 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965336    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965338    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965343    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965346    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965348    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965351    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965353    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965356    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965358    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965361    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965364    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965366    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965369    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965371    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965374    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965376    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965379    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965381    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965384    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:06:35.969874 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965386    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965389    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965391    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965394    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965396    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965399    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965401    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965404    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965406    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965409    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965411    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965414    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965418    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965421    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965424    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965428    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965431    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965433    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965436    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.965438    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965505    2555 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 21:06:35.970369 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965516    2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965528    2555 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965532    2555 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965536    2555 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965539    2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965544    2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965548    2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965552    2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965555    2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965558    2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965561    2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965564    2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965568    2555 flags.go:64] FLAG: --cgroup-root=""
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965570    2555 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965573    2555 flags.go:64] FLAG: --client-ca-file=""
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965577    2555 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965581    2555 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965585    2555 flags.go:64] FLAG: --cluster-domain=""
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965588    2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965591    2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965594    2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965597    2555 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965601    2555 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 21:06:35.970860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965605    2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965608    2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965612    2555 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965616    2555 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965619    2555 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965622    2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965625    2555 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965628    2555 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965632    2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965635    2555 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965638    2555 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965641    2555 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965644    2555 flags.go:64] FLAG: --enable-server="true"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965647    2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965651    2555 flags.go:64] FLAG: --event-burst="100"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965654    2555 flags.go:64] FLAG: --event-qps="50"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965657    2555 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965660    2555 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965663    2555 flags.go:64] FLAG: --eviction-hard=""
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965666    2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965669    2555 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965672    2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965675    2555 flags.go:64] FLAG: --eviction-soft=""
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965678    2555 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965681    2555 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 21:06:35.971413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965684    2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965687    2555 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965690    2555 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965693    2555 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965696    2555 flags.go:64] FLAG: --feature-gates=""
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965699    2555 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965702    2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965707    2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965710    2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965713    2555 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965717    2555 flags.go:64] FLAG: --help="false"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965720    2555 flags.go:64] FLAG: --hostname-override="ip-10-0-128-44.ec2.internal"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965723    2555 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965726    2555 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965729    2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965732    2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965736    2555 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965739    2555 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965741    2555 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965744    2555 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965747    2555 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965750    2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965753    2555 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965756    2555 flags.go:64] FLAG: --kube-reserved=""
Jul 15 21:06:35.972106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965759    2555 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965762    2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965765    2555 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965768    2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965770    2555 flags.go:64] FLAG: --lock-file=""
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965773    2555 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965776    2555 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965779    2555 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965784    2555 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965791    2555 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965793    2555 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965796    2555 flags.go:64] FLAG: --logging-format="text"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965799    2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965802    2555 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965805    2555 flags.go:64] FLAG: --manifest-url=""
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965809    2555 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965814    2555 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965817    2555 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965822    2555 flags.go:64] FLAG: --max-pods="110"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965825    2555 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965828    2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965831    2555 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965834    2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965837    2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965840    2555 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 21:06:35.972694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965842    2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965849    2555 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965853    2555 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965856    2555 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965859    2555 flags.go:64] FLAG: --pod-cidr=""
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965873    2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965879    2555 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965882    2555 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965885    2555 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965888    2555 flags.go:64] FLAG: --port="10250"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965891    2555 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965894    2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00865aa316717ea8f"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965897    2555 flags.go:64] FLAG: --qos-reserved=""
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965900    2555 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965903    2555 flags.go:64] FLAG: --register-node="true"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965906    2555 flags.go:64] FLAG: --register-with-taints=""
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965909    2555 flags.go:64] FLAG: --registry-burst="10"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965916    2555 flags.go:64] FLAG: --registry-qps="5"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965919    2555 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965921    2555 flags.go:64] FLAG: --reserved-memory=""
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965925    2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965928    2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965932    2555 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965935    2555 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965938    2555 flags.go:64] FLAG: --runonce="false"
Jul 15 21:06:35.973316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965940    2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965945    2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965948    2555 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965963    2555 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965967    2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965969    2555 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965972    2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965975    2555 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965979    2555 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965982    2555 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965985    2555 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965988    2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965991    2555 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965994    2555 flags.go:64] FLAG: --system-cgroups=""
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.965997    2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966002    2555 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966005    2555 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966008    2555 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966017    2555 flags.go:64] FLAG: --tls-min-version=""
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966020    2555 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966023    2555 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966026    2555 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966029    2555 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966032    2555 flags.go:64] FLAG: --v="2"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966036    2555 flags.go:64] FLAG: --version="false"
Jul 15 21:06:35.973912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966043    2555 flags.go:64] FLAG: --vmodule=""
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966047    2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.966050    2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966183    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966188    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966192    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966195    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966199    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966205    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966208    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966212    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966215    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966218    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966220    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966223    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966226    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966229    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966231    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966234    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:06:35.974519 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966237    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966239    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966242    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966244    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966247    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966249    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966253    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966256    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966259    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966261    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966264    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966267    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966269    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966272    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966278    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966281    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966284    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966287    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966290    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966292    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:06:35.974986 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966295    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966299    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966302    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966305    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966307    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966310    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966312    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966314    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966317    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966319    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966322    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966325    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966328    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966330    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966332    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966335    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966337    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966340    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966343    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966345    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:06:35.975472 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966347    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966350    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966353    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966355    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966358    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966360    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966362    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966365    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966368    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966372    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966374    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966377    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966380    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966386    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966388    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966391    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966394    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966396    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966400    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:06:35.976004 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966404    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966407    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966409    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966412    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966415    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966418    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966421    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966424    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966427    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966429    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966432    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966434    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966437    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966439    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966442    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966444    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966447    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966450    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966452    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966455    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:06:35.976489 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966457    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966461    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966464    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.966467    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.967167    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.973574    2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.973589    2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.974499    2555 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.974507    2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974565    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974571    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974574    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974577    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974581    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974584    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:06:35.976965 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974587    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974590    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974593    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974595    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974598    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974600    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974603    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974606    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974608    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974611    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974613    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974616    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974619    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974621    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974624    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974626    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974629    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974631    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974634    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974637    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:06:35.977328 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974640    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974642    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974645    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974649    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974653    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974656    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974659    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974662    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974666    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974669    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974672    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974675    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974678    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974680    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974683    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974685    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974688    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974690    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:06:35.977847 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974693    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974696    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974698    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974701    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974703    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974706    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974709    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974711    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974713    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974718    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974721    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974725    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974728    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974731    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974734    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974737    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974740    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974742    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974745    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:06:35.978340 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974747    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974750    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974752    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974755    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974757    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974760    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974762    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974765    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974767    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974770    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974772    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974774    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974777    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974779    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974782    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974784    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974786    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974789    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974792    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974794    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:06:35.978848 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974797    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974799    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974802    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974805    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974808    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974810    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974813    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974816    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974818    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974821    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974823    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974826    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974829    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974831    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974834    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:06:35.979338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974836    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.974841    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974946    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974950    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974966    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974970    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974973    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974976    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974979    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974982    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974985    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974987    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974990    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974993    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974996    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:06:35.979838 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.974998    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975002    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975006    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975008    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975011    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975014    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975016    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975019    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975021    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975024    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975027    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975030    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975033    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975035    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975038    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975040    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975043    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975045    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975048    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:06:35.980305 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975051    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975053    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975056    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975058    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975061    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975063    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975066    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975068    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975071    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975073    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975076    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975078    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975081    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975083    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975085    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975088    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975090    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975093    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975096    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975098    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975101    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:06:35.980749 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975103    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975106    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975108    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975111    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975115    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975119    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975121    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975124    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975127    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975129    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975133    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975136    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975139    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975142    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975145    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975148    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975150    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975153    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975155    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:06:35.981551 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975157    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975160    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975162    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975165    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975167    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975169    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975172    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975174    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975177    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975180    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975182    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975185    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975187    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975189    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975192    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975194    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975197    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975200    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975202    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975205    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:06:35.982332 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975207    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975210    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975212    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975215    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975217    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975219    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:35.975222    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.975227    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.975321    2555 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.977505    2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.978715    2555 server.go:1019] "Starting client certificate rotation"
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.978823    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.979395    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 21:06:35.983150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.981527    2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 21:06:35.985799 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.985780    2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 21:06:35.998054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:35.998037    2555 log.go:25] "Validated CRI v1 runtime API"
Jul 15 21:06:36.000076 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.000063    2555 log.go:25] "Validated CRI v1 image API"
Jul 15 21:06:36.001018 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.000990    2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 21:06:36.005084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.005059    2555 fs.go:135] Filesystem UUIDs: map[3deb1b40-2b7b-46dd-8aa0-55826a783f4a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d8c5f89c-c625-4ff2-a59d-f707a15a8ff2:/dev/nvme0n1p3]
Jul 15 21:06:36.005144 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.005082    2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 21:06:36.007975 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.007940    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 21:06:36.010541 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.010433    2555 manager.go:216] Machine: {Timestamp:2026-07-15 21:06:36.008643019 +0000 UTC m=+0.340784111 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099666 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2287ca99e2a84da71a27788acd6a87 SystemUUID:ec2287ca-99e2-a84d-a71a-27788acd6a87 BootID:4993bc32-3605-4061-a4d8-a287a32eb2f9 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:f8:7d:ca:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:f8:7d:ca:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:3c:a7:62:ac:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 21:06:36.010541 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.010532    2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 21:06:36.010654 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.010630    2555 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 21:06:36.014283 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.014257    2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 21:06:36.014422 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.014291    2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-44.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 21:06:36.015002 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.014992    2555 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 21:06:36.015032 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.015005    2555 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 21:06:36.015032 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.015013    2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 21:06:36.015612 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.015602    2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 21:06:36.016296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.016287    2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 21:06:36.019293 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019282    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 21:06:36.019412 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019402    2555 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 21:06:36.019451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019431    2555 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 21:06:36.019451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019450    2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 21:06:36.019521 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019489    2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 21:06:36.019521 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019504    2555 kubelet.go:398] "Adding apiserver pod source"
Jul 15 21:06:36.019521 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019513    2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 21:06:36.019918 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019905    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 21:06:36.019918 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.019927    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 21:06:36.023064 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.023042    2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 21:06:36.024502 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.024484    2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 21:06:36.024546 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.024519    2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 21:06:36.025269 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025243    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 21:06:36.025269 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025272    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025286    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025299    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025316    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025328    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025345    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025356    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025378    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 21:06:36.025389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025388    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 21:06:36.025592 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025405    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 21:06:36.025592 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.025416    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 21:06:36.026877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.026864    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 21:06:36.026877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.026876    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 21:06:36.028468 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.028448    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ln4mp"
Jul 15 21:06:36.029733 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.029712    2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-44.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 21:06:36.029813 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.029753    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 21:06:36.029813 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.029796    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 21:06:36.030244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.030231    2555 server.go:1268] "Started kubelet"
Jul 15 21:06:36.030308 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.030290    2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 21:06:36.030434 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.030374    2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 21:06:36.030492 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.030457    2555 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 21:06:36.031126 ip-10-0-128-44 systemd[1]: Started Kubernetes Kubelet.
Jul 15 21:06:36.034293 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.034257    2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 21:06:36.035337 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.035321    2555 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 21:06:36.037143 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.037124    2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 21:06:36.037949 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.037935    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ln4mp"
Jul 15 21:06:36.041026 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041012    2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 21:06:36.041084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041068    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 21:06:36.041525 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.039523    2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-44.ec2.internal.18c2929ad189762a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-44.ec2.internal,UID:ip-10-0-128-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-44.ec2.internal,},FirstTimestamp:2026-07-15 21:06:36.030203434 +0000 UTC m=+0.362344515,LastTimestamp:2026-07-15 21:06:36.030203434 +0000 UTC m=+0.362344515,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-44.ec2.internal,}"
Jul 15 21:06:36.041525 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.041245    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.041848 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041834    2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 21:06:36.041848 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041834    2555 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 21:06:36.042005 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041868    2555 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 21:06:36.042005 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041919    2555 factory.go:55] Registering systemd factory
Jul 15 21:06:36.042005 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041935    2555 factory.go:223] Registration of the systemd container factory successfully
Jul 15 21:06:36.042560 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.041946    2555 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 21:06:36.042627 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.042570    2555 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 21:06:36.042627 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.042148    2555 factory.go:153] Registering CRI-O factory
Jul 15 21:06:36.042627 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.042595    2555 factory.go:223] Registration of the crio container factory successfully
Jul 15 21:06:36.042753 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.042660    2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 21:06:36.042753 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.042685    2555 factory.go:103] Registering Raw factory
Jul 15 21:06:36.042753 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.042715    2555 manager.go:1196] Started watching for new ooms in manager
Jul 15 21:06:36.043093 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.043081    2555 manager.go:349] Starting recovery of all containers
Jul 15 21:06:36.048748 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.048728    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:06:36.053056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.053042    2555 manager.go:354] Recovery completed
Jul 15 21:06:36.053498 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.053479    2555 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-44.ec2.internal\" not found" node="ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.057717 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.057705    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.059451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059429    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.059524 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059457    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.059524 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059466    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.059963 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059936    2555 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 21:06:36.059963 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059944    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.060055 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059950    2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 21:06:36.060055 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.059993    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 21:06:36.060613 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.060597    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.060679 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.060625    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.060679 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.060637    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.061101 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.061088    2555 policy_none.go:49] "None policy: Start"
Jul 15 21:06:36.061168 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.061104    2555 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 21:06:36.061168 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.061115    2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 21:06:36.062139 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.062128    2555 policy_none.go:47] "Start"
Jul 15 21:06:36.104137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104029    2555 manager.go:337] "Starting Device Plugin manager"
Jul 15 21:06:36.104226 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.104147    2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 21:06:36.104226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104161    2555 server.go:87] "Starting device plugin registration server"
Jul 15 21:06:36.104375 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104361    2555 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 21:06:36.104431 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104381    2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 21:06:36.104476 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104467    2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 21:06:36.104564 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104547    2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 21:06:36.104564 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.104561    2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 21:06:36.105019 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.105003    2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 21:06:36.105093 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.105047    2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.205434 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.205368    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.206767 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.206391    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.206767 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.206420    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.206767 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.206430    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.206767 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.206455    2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.209623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.209597    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 21:06:36.210732 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.210716    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 21:06:36.210816 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.210761    2555 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 21:06:36.210816 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.210787    2555 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 21:06:36.210897 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.210829    2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 21:06:36.213397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.213373    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:06:36.215426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.215412    2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.215504 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.215432    2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-44.ec2.internal\": node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.263613 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.263593    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.311696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.311667    2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"]
Jul 15 21:06:36.311751 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.311735    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.313296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.313280    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.313353 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.313309    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.313353 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.313326    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.314587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.314574    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.314738 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.314725    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.314774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.314753    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.315246 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.315230    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.315325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.315255    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.315325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.315270    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.315325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.315281    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.315325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.315297    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.315325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.315308    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.316889 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.316867    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.316982 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.316895    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:06:36.317525 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.317511    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:06:36.317575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.317537    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:06:36.317575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.317551    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:06:36.342599 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.342582    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-44.ec2.internal\" not found" node="ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.344761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.344742    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d59bb19fbb9fb26344a0b82fdd45c82a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal\" (UID: \"d59bb19fbb9fb26344a0b82fdd45c82a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.344826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.344776    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d59bb19fbb9fb26344a0b82fdd45c82a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal\" (UID: \"d59bb19fbb9fb26344a0b82fdd45c82a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.344826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.344803    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72a16374957430e777a4eeb37d6601d6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-44.ec2.internal\" (UID: \"72a16374957430e777a4eeb37d6601d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.346931 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.346917    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-44.ec2.internal\" not found" node="ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.364455 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.364438    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.445384 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.445362    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d59bb19fbb9fb26344a0b82fdd45c82a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal\" (UID: \"d59bb19fbb9fb26344a0b82fdd45c82a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.445465 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.445388    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d59bb19fbb9fb26344a0b82fdd45c82a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal\" (UID: \"d59bb19fbb9fb26344a0b82fdd45c82a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.445465 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.445417    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72a16374957430e777a4eeb37d6601d6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-44.ec2.internal\" (UID: \"72a16374957430e777a4eeb37d6601d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.445528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.445465    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d59bb19fbb9fb26344a0b82fdd45c82a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal\" (UID: \"d59bb19fbb9fb26344a0b82fdd45c82a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.445528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.445476    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/72a16374957430e777a4eeb37d6601d6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-44.ec2.internal\" (UID: \"72a16374957430e777a4eeb37d6601d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.445528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.445523    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d59bb19fbb9fb26344a0b82fdd45c82a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal\" (UID: \"d59bb19fbb9fb26344a0b82fdd45c82a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.464665 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.464616    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.565380 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.565350    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.645785 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.645765    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.650858 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.650791    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:36.665887 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.665867    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.766393 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.766324    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.866838 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.866816    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.967252 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:36.967227    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:36.978402 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.978373    2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 21:06:36.978520 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.978501    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 21:06:36.978582 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:36.978554    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 21:06:37.041760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.041691    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 21:01:36 +0000 UTC" deadline="2028-04-21 20:11:58.015595622 +0000 UTC"
Jul 15 21:06:37.041760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.041717    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15503h5m20.973881948s"
Jul 15 21:06:37.067310 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:37.067284    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-44.ec2.internal\" not found"
Jul 15 21:06:37.067723 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.067707    2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:06:37.142053 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.142028    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:37.154045 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.154022    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 21:06:37.156205 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.156191    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal"
Jul 15 21:06:37.166219 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.166199    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 21:06:37.311366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.311311    2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:06:37.367815 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:37.367778    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a16374957430e777a4eeb37d6601d6.slice/crio-731a4b9dcfc662c65424a2519dbede1c3943051f8ee18f875761b597f750c791 WatchSource:0}: Error finding container 731a4b9dcfc662c65424a2519dbede1c3943051f8ee18f875761b597f750c791: Status 404 returned error can't find the container with id 731a4b9dcfc662c65424a2519dbede1c3943051f8ee18f875761b597f750c791
Jul 15 21:06:37.373274 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.373255    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:06:37.889011 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:37.888946    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59bb19fbb9fb26344a0b82fdd45c82a.slice/crio-d4fdddd9e99106a8b0fd3b6b13cc18927f2f0ad5fd88af47a5cf24a19aef5df5 WatchSource:0}: Error finding container d4fdddd9e99106a8b0fd3b6b13cc18927f2f0ad5fd88af47a5cf24a19aef5df5: Status 404 returned error can't find the container with id d4fdddd9e99106a8b0fd3b6b13cc18927f2f0ad5fd88af47a5cf24a19aef5df5
Jul 15 21:06:37.925211 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:37.925174    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:06:38.021222 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.021196    2555 apiserver.go:52] "Watching apiserver"
Jul 15 21:06:38.032525 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.032108    2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 21:06:38.034086 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.034049    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zccbq","kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9hhxl","openshift-dns/node-resolver-jsw54","openshift-image-registry/node-ca-qms62","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal","openshift-multus/multus-additional-cni-plugins-vjmbd","openshift-network-operator/iptables-alerter-9f58w","kube-system/konnectivity-agent-mnpcr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk","openshift-multus/multus-nbgw5","openshift-multus/network-metrics-daemon-htj74","openshift-network-diagnostics/network-check-target-wgwt4","openshift-ovn-kubernetes/ovnkube-node-4r92m"]
Jul 15 21:06:38.035901 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.035878    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.037360 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.037340    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:38.037476 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.037408    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:38.038970 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.038936    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 21:06:38.039079 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.038937    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pl8bk\""
Jul 15 21:06:38.039393 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.039351    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.039621 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.039602    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 21:06:38.039694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.039632    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 21:06:38.039795 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.039774    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.039903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.039843    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.040821 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.040803    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.042441 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.042422    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 21:06:38.042643 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.042625    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7z6xl\""
Jul 15 21:06:38.042941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.042924    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.043199 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.043180    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.043397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.043380    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.043470 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.043384    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.043822 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.043804    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.044049 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.044033    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-29sb8\""
Jul 15 21:06:38.044466 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.044212    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.045269 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.045248    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.046040 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.046023    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.046238 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.046134    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 21:06:38.046481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.046189    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.046481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.046226    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lqgqh\""
Jul 15 21:06:38.046624 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.046613    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.046703 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.046681    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:38.048489 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.047988    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.049260 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.049240    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.049943 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.049782    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 21:06:38.049943 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.049885    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.050132 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050106    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.050189 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050170    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 21:06:38.050486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050241    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-djw8k\""
Jul 15 21:06:38.050486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050351    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 21:06:38.050486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050116    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 21:06:38.050486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050353    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-52kq7\""
Jul 15 21:06:38.050715 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050607    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 21:06:38.050715 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050647    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.051009 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.050991    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 21:06:38.051369 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.051349    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.051577    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.051835    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zvl5s\""
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.051981    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/200938e3-b583-4a3c-9bdb-fcbc25af1105-hosts-file\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052015    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/200938e3-b583-4a3c-9bdb-fcbc25af1105-tmp-dir\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052040    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-sys\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052050    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052063    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-os-release\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052102    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-var-lib-kubelet\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052133    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16207249-d9af-4610-a473-fc018f680361-serviceca\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052160    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pklj5\" (UniqueName: \"kubernetes.io/projected/16207249-d9af-4610-a473-fc018f680361-kube-api-access-pklj5\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052188    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysconfig\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052210    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-run\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052247    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-modprobe-d\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052261    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052269    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysctl-d\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052274    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052291    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052314    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-kubernetes\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.052366 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.052330    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052336    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-host\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052385    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fc5605d-bebc-4f92-923a-969cfb3e3d79-tmp\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052419    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16207249-d9af-4610-a473-fc018f680361-host\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052453    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-system-cni-dir\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052462    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fwwg6\""
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052477    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cnibin\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052507    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052532    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xb6k\" (UniqueName: \"kubernetes.io/projected/b1ebae01-6922-4cb9-a266-0bedf8811e6a-kube-api-access-8xb6k\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052556    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysctl-conf\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052577    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-lib-modules\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052599    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052631    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8w9r\" (UniqueName: \"kubernetes.io/projected/1fc5605d-bebc-4f92-923a-969cfb3e3d79-kube-api-access-n8w9r\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052654    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhj7x\" (UniqueName: \"kubernetes.io/projected/200938e3-b583-4a3c-9bdb-fcbc25af1105-kube-api-access-fhj7x\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052679    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052702    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052739    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-systemd\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.053301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052781    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-tuned\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.054226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052805    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.054226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.052868    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 21:06:38.054226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.053120    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 21:06:38.054226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.053755    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qfj9h\""
Jul 15 21:06:38.060231 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.060209    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 21:06:38.084263 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.084246    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dpx56"
Jul 15 21:06:38.101397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.101360    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dpx56"
Jul 15 21:06:38.143540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.143488    2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 21:06:38.153326 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153301    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-hostroot\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.153416 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153343    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/200938e3-b583-4a3c-9bdb-fcbc25af1105-hosts-file\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.153416 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153371    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/200938e3-b583-4a3c-9bdb-fcbc25af1105-tmp-dir\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.153416 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153394    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153420    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxvw\" (UniqueName: \"kubernetes.io/projected/85aa708f-860f-4dcd-bcbd-93511f697a9c-kube-api-access-kbxvw\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153447    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-os-release\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153445    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/200938e3-b583-4a3c-9bdb-fcbc25af1105-hosts-file\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153486    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-os-release\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153518    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74396062-85c2-4f03-bfb9-48ce98050e33-multus-daemon-config\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153541    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-os-release\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.153563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153546    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-var-lib-kubelet\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153593    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-var-lib-kubelet\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153592    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16207249-d9af-4610-a473-fc018f680361-serviceca\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153631    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklj5\" (UniqueName: \"kubernetes.io/projected/16207249-d9af-4610-a473-fc018f680361-kube-api-access-pklj5\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153649    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/200938e3-b583-4a3c-9bdb-fcbc25af1105-tmp-dir\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153650    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmrvf\" (UniqueName: \"kubernetes.io/projected/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-kube-api-access-qmrvf\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153714    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-ovn\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153742    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj865\" (UniqueName: \"kubernetes.io/projected/74396062-85c2-4f03-bfb9-48ce98050e33-kube-api-access-kj865\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153770    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysconfig\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153792    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-run\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153815    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-log-socket\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153841    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h99wd\" (UniqueName: \"kubernetes.io/projected/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-kube-api-access-h99wd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.153888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153884    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74396062-85c2-4f03-bfb9-48ce98050e33-cni-binary-copy\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153904    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysconfig\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153908    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysctl-d\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.153979    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154008    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-cni-multus\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154011    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16207249-d9af-4610-a473-fc018f680361-serviceca\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154034    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-kubernetes\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154051    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-run\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154059    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fc5605d-bebc-4f92-923a-969cfb3e3d79-tmp\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154085    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16207249-d9af-4610-a473-fc018f680361-host\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154122    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16207249-d9af-4610-a473-fc018f680361-host\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154115    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cnibin\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154091    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-kubernetes\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154143    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysctl-d\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154168    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-systemd-units\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154207    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-sys-fs\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154230    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysctl-conf\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.154480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154180    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cnibin\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154269    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w9r\" (UniqueName: \"kubernetes.io/projected/1fc5605d-bebc-4f92-923a-969cfb3e3d79-kube-api-access-n8w9r\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154300    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-host-slash\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154312    2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154326    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-run-netns\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154350    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-cni-netd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154368    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovnkube-config\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154386    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-cnibin\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154405    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-conf-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154411    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-sysctl-conf\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154429    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-systemd\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154455    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154503    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154523    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-systemd\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154557    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154587    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-system-cni-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154619    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-iptables-alerter-script\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.155316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154655    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154680    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-cni-bin\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154708    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-env-overrides\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154731    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovn-node-metrics-cert\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154767    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-sys\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154795    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2kmn\" (UniqueName: \"kubernetes.io/projected/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-kube-api-access-r2kmn\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154818    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-kubelet\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154831    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-sys\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154840    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-slash\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154868    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-run-ovn-kubernetes\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154892    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-device-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154913    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-kubelet\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154937    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f5324-3152-4e5c-b9d3-57007a007d8c-konnectivity-ca\") pod \"konnectivity-agent-mnpcr\" (UID: \"ab1f5324-3152-4e5c-b9d3-57007a007d8c\") " pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154976    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/543fcfc2-8dba-49bd-a5f4-5a9064532860-kubelet-config\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.154996    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-registration-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155015    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-multus-certs\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.155964 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155034    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155067    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155101    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovnkube-script-lib\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155127    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-socket-dir-parent\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155148    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-etc-kubernetes\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155167    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-modprobe-d\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155197    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab1f5324-3152-4e5c-b9d3-57007a007d8c-agent-certs\") pod \"konnectivity-agent-mnpcr\" (UID: \"ab1f5324-3152-4e5c-b9d3-57007a007d8c\") " pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155245    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-systemd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155256    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-modprobe-d\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155274    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-node-log\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155300    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-host\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155325    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-system-cni-dir\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155344    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-host\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155353    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155381    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xb6k\" (UniqueName: \"kubernetes.io/projected/b1ebae01-6922-4cb9-a266-0bedf8811e6a-kube-api-access-8xb6k\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155404    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-var-lib-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.156655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155384    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1ebae01-6922-4cb9-a266-0bedf8811e6a-system-cni-dir\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155428    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155460    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-cni-bin\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155486    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-lib-modules\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155513    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhj7x\" (UniqueName: \"kubernetes.io/projected/200938e3-b583-4a3c-9bdb-fcbc25af1105-kube-api-access-fhj7x\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155537    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155561    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155586    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-etc-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155646    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155676    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-socket-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155702    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-cni-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155728    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-tuned\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155751    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/543fcfc2-8dba-49bd-a5f4-5a9064532860-dbus\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155773    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-k8s-cni-cncf-io\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155794    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-netns\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155843    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fc5605d-bebc-4f92-923a-969cfb3e3d79-lib-modules\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.157259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.155843    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.157844 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.156278    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1ebae01-6922-4cb9-a266-0bedf8811e6a-cni-binary-copy\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.158174 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.158159    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1fc5605d-bebc-4f92-923a-969cfb3e3d79-etc-tuned\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.158601 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.158589    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fc5605d-bebc-4f92-923a-969cfb3e3d79-tmp\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.164415 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.164397    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:06:38.164415 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.164419    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:06:38.164566 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.164432    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:38.164566 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.164497    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:06:38.664477945 +0000 UTC m=+2.996619030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:38.166973 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.166931    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklj5\" (UniqueName: \"kubernetes.io/projected/16207249-d9af-4610-a473-fc018f680361-kube-api-access-pklj5\") pod \"node-ca-qms62\" (UID: \"16207249-d9af-4610-a473-fc018f680361\") " pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.167067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.167022    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xb6k\" (UniqueName: \"kubernetes.io/projected/b1ebae01-6922-4cb9-a266-0bedf8811e6a-kube-api-access-8xb6k\") pod \"multus-additional-cni-plugins-vjmbd\" (UID: \"b1ebae01-6922-4cb9-a266-0bedf8811e6a\") " pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.167120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.167067    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhj7x\" (UniqueName: \"kubernetes.io/projected/200938e3-b583-4a3c-9bdb-fcbc25af1105-kube-api-access-fhj7x\") pod \"node-resolver-jsw54\" (UID: \"200938e3-b583-4a3c-9bdb-fcbc25af1105\") " pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.167481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.167458    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8w9r\" (UniqueName: \"kubernetes.io/projected/1fc5605d-bebc-4f92-923a-969cfb3e3d79-kube-api-access-n8w9r\") pod \"tuned-9hhxl\" (UID: \"1fc5605d-bebc-4f92-923a-969cfb3e3d79\") " pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.200891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.200863    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:06:38.217520 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.217465    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal" event={"ID":"d59bb19fbb9fb26344a0b82fdd45c82a","Type":"ContainerStarted","Data":"d4fdddd9e99106a8b0fd3b6b13cc18927f2f0ad5fd88af47a5cf24a19aef5df5"}
Jul 15 21:06:38.218599 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.218571    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal" event={"ID":"72a16374957430e777a4eeb37d6601d6","Type":"ContainerStarted","Data":"731a4b9dcfc662c65424a2519dbede1c3943051f8ee18f875761b597f750c791"}
Jul 15 21:06:38.256593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256565    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-socket-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.256735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256602    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-cni-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.256735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256628    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/543fcfc2-8dba-49bd-a5f4-5a9064532860-dbus\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.256735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256664    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-k8s-cni-cncf-io\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.256735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256692    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-netns\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.256735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256712    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-hostroot\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256737    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256738    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-k8s-cni-cncf-io\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256761    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxvw\" (UniqueName: \"kubernetes.io/projected/85aa708f-860f-4dcd-bcbd-93511f697a9c-kube-api-access-kbxvw\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256764    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-cni-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256762    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/543fcfc2-8dba-49bd-a5f4-5a9064532860-dbus\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256800    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-os-release\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256819    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256817    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-netns\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256830    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74396062-85c2-4f03-bfb9-48ce98050e33-multus-daemon-config\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256871    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-os-release\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256878    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-hostroot\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256884    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmrvf\" (UniqueName: \"kubernetes.io/projected/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-kube-api-access-qmrvf\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256849    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-socket-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256912    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-ovn\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256937    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj865\" (UniqueName: \"kubernetes.io/projected/74396062-85c2-4f03-bfb9-48ce98050e33-kube-api-access-kj865\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-log-socket\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.256996    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-ovn\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257001    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h99wd\" (UniqueName: \"kubernetes.io/projected/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-kube-api-access-h99wd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257052    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74396062-85c2-4f03-bfb9-48ce98050e33-cni-binary-copy\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257087    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-cni-multus\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257116    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-systemd-units\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257145    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-log-socket\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257146    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-sys-fs\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257171    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-host-slash\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257186    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-run-netns\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257188    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-sys-fs\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257201    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-cni-netd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257227    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovnkube-config\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257257    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-host-slash\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257255    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-cnibin\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257287    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-conf-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257293    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-systemd-units\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257304    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-cnibin\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257229    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-cni-netd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257256    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-cni-multus\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.257687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257313    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257348    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-run-netns\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257379    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-conf-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257414    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257419    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/74396062-85c2-4f03-bfb9-48ce98050e33-multus-daemon-config\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257416    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-system-cni-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257474    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-iptables-alerter-script\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257490    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-cni-bin\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257507    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-env-overrides\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257518    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/74396062-85c2-4f03-bfb9-48ce98050e33-cni-binary-copy\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257530    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovn-node-metrics-cert\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257552    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-system-cni-dir\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257560    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2kmn\" (UniqueName: \"kubernetes.io/projected/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-kube-api-access-r2kmn\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257566    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-cni-bin\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257583    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-kubelet\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257607    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-slash\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257629    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-run-ovn-kubernetes\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.258593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257665    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-kubelet\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257653    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-device-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257707    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-kubelet\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257734    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f5324-3152-4e5c-b9d3-57007a007d8c-konnectivity-ca\") pod \"konnectivity-agent-mnpcr\" (UID: \"ab1f5324-3152-4e5c-b9d3-57007a007d8c\") " pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257746    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-run-ovn-kubernetes\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257708    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-device-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257758    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/543fcfc2-8dba-49bd-a5f4-5a9064532860-kubelet-config\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257783    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-registration-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257796    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-kubelet\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257810    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-multus-certs\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257836    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257838    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/543fcfc2-8dba-49bd-a5f4-5a9064532860-kubelet-config\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257867    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257878    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-slash\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257895    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovnkube-script-lib\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257918    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-socket-dir-parent\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257938    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovnkube-config\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.259380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.257945    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-etc-kubernetes\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258005    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab1f5324-3152-4e5c-b9d3-57007a007d8c-agent-certs\") pod \"konnectivity-agent-mnpcr\" (UID: \"ab1f5324-3152-4e5c-b9d3-57007a007d8c\") " pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258029    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-systemd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.258056    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258066    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-node-log\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258072    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-multus-socket-dir-parent\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258095    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-var-lib-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258116    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258118    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.258136    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:38.758116071 +0000 UTC m=+3.090257165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258026    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-registration-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258166    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85aa708f-860f-4dcd-bcbd-93511f697a9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258216    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-etc-kubernetes\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258228    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-cni-bin\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258274    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-etc-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258301    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258365    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-etc-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260058 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258368    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-var-lib-cni-bin\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258027    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-iptables-alerter-script\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258410    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ab1f5324-3152-4e5c-b9d3-57007a007d8c-konnectivity-ca\") pod \"konnectivity-agent-mnpcr\" (UID: \"ab1f5324-3152-4e5c-b9d3-57007a007d8c\") " pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258416    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-run-systemd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.258463    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258469    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-var-lib-openvswitch\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258469    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-node-log\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258504    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/74396062-85c2-4f03-bfb9-48ce98050e33-host-run-multus-certs\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.258513    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:38.758497096 +0000 UTC m=+3.090638178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258586    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-env-overrides\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.258835    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovnkube-script-lib\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.260706 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.260636    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ab1f5324-3152-4e5c-b9d3-57007a007d8c-agent-certs\") pod \"konnectivity-agent-mnpcr\" (UID: \"ab1f5324-3152-4e5c-b9d3-57007a007d8c\") " pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.261153 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.260949    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-ovn-node-metrics-cert\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.271438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.271329    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmrvf\" (UniqueName: \"kubernetes.io/projected/0378e56f-0e25-4ef6-96be-9e96ed50ef7d-kube-api-access-qmrvf\") pod \"iptables-alerter-9f58w\" (UID: \"0378e56f-0e25-4ef6-96be-9e96ed50ef7d\") " pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.271739 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.271719    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbxvw\" (UniqueName: \"kubernetes.io/projected/85aa708f-860f-4dcd-bcbd-93511f697a9c-kube-api-access-kbxvw\") pod \"aws-ebs-csi-driver-node-msjpk\" (UID: \"85aa708f-860f-4dcd-bcbd-93511f697a9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.271823 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.271801    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h99wd\" (UniqueName: \"kubernetes.io/projected/a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4-kube-api-access-h99wd\") pod \"ovnkube-node-4r92m\" (UID: \"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4\") " pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.271877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.271838    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2kmn\" (UniqueName: \"kubernetes.io/projected/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-kube-api-access-r2kmn\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.273011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.272973    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj865\" (UniqueName: \"kubernetes.io/projected/74396062-85c2-4f03-bfb9-48ce98050e33-kube-api-access-kj865\") pod \"multus-nbgw5\" (UID: \"74396062-85c2-4f03-bfb9-48ce98050e33\") " pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.349124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.349091    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vjmbd"
Jul 15 21:06:38.360064 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.360023    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9hhxl"
Jul 15 21:06:38.368786 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.368745    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jsw54"
Jul 15 21:06:38.375765 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.375615    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qms62"
Jul 15 21:06:38.381380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.381337    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:06:38.389267 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.389248    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:06:38.395284 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.395234    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9f58w"
Jul 15 21:06:38.401995 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.401975    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk"
Jul 15 21:06:38.407684 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.407591    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nbgw5"
Jul 15 21:06:38.761926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.761837    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:38.761926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.761892    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:38.761926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:38.761912    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762034    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762060    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762085    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762097    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762104    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:39.762084862 +0000 UTC m=+4.094225948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762155    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:06:39.762138244 +0000 UTC m=+4.094279336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762061    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:38.762219 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:38.762209    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:39.762198171 +0000 UTC m=+4.094339241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:39.103306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:39.103230    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 21:01:38 +0000 UTC" deadline="2027-12-19 12:05:01.806114696 +0000 UTC"
Jul 15 21:06:39.103306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:39.103255    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12518h58m22.702863007s"
Jul 15 21:06:39.273505 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.272905    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ee6aff_bea5_4e6f_a88a_fd11d8a4cee4.slice/crio-a83704d35cd540dc2b72693768386ad071476a07d4c04d89cce192e5440dae33 WatchSource:0}: Error finding container a83704d35cd540dc2b72693768386ad071476a07d4c04d89cce192e5440dae33: Status 404 returned error can't find the container with id a83704d35cd540dc2b72693768386ad071476a07d4c04d89cce192e5440dae33
Jul 15 21:06:39.277758 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.277731    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc5605d_bebc_4f92_923a_969cfb3e3d79.slice/crio-6a9073a5ff7a3fe1c408b21a413aa0e19866a01dd8c4c00f3c4df251673bdfb3 WatchSource:0}: Error finding container 6a9073a5ff7a3fe1c408b21a413aa0e19866a01dd8c4c00f3c4df251673bdfb3: Status 404 returned error can't find the container with id 6a9073a5ff7a3fe1c408b21a413aa0e19866a01dd8c4c00f3c4df251673bdfb3
Jul 15 21:06:39.278368 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.278345    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0378e56f_0e25_4ef6_96be_9e96ed50ef7d.slice/crio-4af394312633946e42d2238d8ab058c2cc8cfabe31be2a663689ebaff521fd8c WatchSource:0}: Error finding container 4af394312633946e42d2238d8ab058c2cc8cfabe31be2a663689ebaff521fd8c: Status 404 returned error can't find the container with id 4af394312633946e42d2238d8ab058c2cc8cfabe31be2a663689ebaff521fd8c
Jul 15 21:06:39.279146 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.279124    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ebae01_6922_4cb9_a266_0bedf8811e6a.slice/crio-1d92f1017375b8e07ede5d3ac59859294d609ed0c488afbc8e7d10d33c69402b WatchSource:0}: Error finding container 1d92f1017375b8e07ede5d3ac59859294d609ed0c488afbc8e7d10d33c69402b: Status 404 returned error can't find the container with id 1d92f1017375b8e07ede5d3ac59859294d609ed0c488afbc8e7d10d33c69402b
Jul 15 21:06:39.279804 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.279728    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200938e3_b583_4a3c_9bdb_fcbc25af1105.slice/crio-18642ccfcab3541d7cca2a536c518638aeda9ebf34d3fa4d08b523c1cc537014 WatchSource:0}: Error finding container 18642ccfcab3541d7cca2a536c518638aeda9ebf34d3fa4d08b523c1cc537014: Status 404 returned error can't find the container with id 18642ccfcab3541d7cca2a536c518638aeda9ebf34d3fa4d08b523c1cc537014
Jul 15 21:06:39.281185 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.281116    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85aa708f_860f_4dcd_bcbd_93511f697a9c.slice/crio-efd62f1fe81ceb83ebcf11e05604eabc1ebcf1128e42e5311b1c7ad938768f73 WatchSource:0}: Error finding container efd62f1fe81ceb83ebcf11e05604eabc1ebcf1128e42e5311b1c7ad938768f73: Status 404 returned error can't find the container with id efd62f1fe81ceb83ebcf11e05604eabc1ebcf1128e42e5311b1c7ad938768f73
Jul 15 21:06:39.283418 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.283346    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16207249_d9af_4610_a473_fc018f680361.slice/crio-6fe8ebac29a44f5b3583b8df5d0943266de7b35358a8496fa866fba7e5693fe0 WatchSource:0}: Error finding container 6fe8ebac29a44f5b3583b8df5d0943266de7b35358a8496fa866fba7e5693fe0: Status 404 returned error can't find the container with id 6fe8ebac29a44f5b3583b8df5d0943266de7b35358a8496fa866fba7e5693fe0
Jul 15 21:06:39.283849 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:06:39.283828    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1f5324_3152_4e5c_b9d3_57007a007d8c.slice/crio-9c01e05ab7d5ae3ba0e87eca5d0e84e3d00e582bb1ca4599f4e9e0b897d47352 WatchSource:0}: Error finding container 9c01e05ab7d5ae3ba0e87eca5d0e84e3d00e582bb1ca4599f4e9e0b897d47352: Status 404 returned error can't find the container with id 9c01e05ab7d5ae3ba0e87eca5d0e84e3d00e582bb1ca4599f4e9e0b897d47352
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:39.767799    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:39.768081    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768142    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:39.768161    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768171    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768186    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768246    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:06:41.768226566 +0000 UTC m=+6.100367646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768291    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768339    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:41.76832472 +0000 UTC m=+6.100465793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768398    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:39.768532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:39.768429    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:41.76841878 +0000 UTC m=+6.100559854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:40.104232 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.104117    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 21:01:38 +0000 UTC" deadline="2028-01-22 23:26:12.260020937 +0000 UTC"
Jul 15 21:06:40.104232 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.104182    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13346h19m32.155843289s"
Jul 15 21:06:40.213927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.213897    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:40.214087 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:40.214054    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:40.214474 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.214453    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:40.214567 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:40.214548    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:40.214629 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.214612    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:40.214696 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:40.214678    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:40.236808 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.236781    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerStarted","Data":"1d92f1017375b8e07ede5d3ac59859294d609ed0c488afbc8e7d10d33c69402b"}
Jul 15 21:06:40.241947 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.241885    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9f58w" event={"ID":"0378e56f-0e25-4ef6-96be-9e96ed50ef7d","Type":"ContainerStarted","Data":"4af394312633946e42d2238d8ab058c2cc8cfabe31be2a663689ebaff521fd8c"}
Jul 15 21:06:40.245966 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.245880    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk" event={"ID":"85aa708f-860f-4dcd-bcbd-93511f697a9c","Type":"ContainerStarted","Data":"efd62f1fe81ceb83ebcf11e05604eabc1ebcf1128e42e5311b1c7ad938768f73"}
Jul 15 21:06:40.248331 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.248279    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9hhxl" event={"ID":"1fc5605d-bebc-4f92-923a-969cfb3e3d79","Type":"ContainerStarted","Data":"6a9073a5ff7a3fe1c408b21a413aa0e19866a01dd8c4c00f3c4df251673bdfb3"}
Jul 15 21:06:40.252837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.252735    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"a83704d35cd540dc2b72693768386ad071476a07d4c04d89cce192e5440dae33"}
Jul 15 21:06:40.272320 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.272262    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nbgw5" event={"ID":"74396062-85c2-4f03-bfb9-48ce98050e33","Type":"ContainerStarted","Data":"8b97c26c81f771e2910e044630efdbb6dc811abae78820eafe346432a7714d36"}
Jul 15 21:06:40.283037 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.282706    2555 generic.go:356] "Generic (PLEG): container finished" podID="d59bb19fbb9fb26344a0b82fdd45c82a" containerID="45766ec302650e39d39a16e9b3ef4ad02e0a637189e11a3b184a6c8b46221d86" exitCode=0
Jul 15 21:06:40.283037 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.282794    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal" event={"ID":"d59bb19fbb9fb26344a0b82fdd45c82a","Type":"ContainerDied","Data":"45766ec302650e39d39a16e9b3ef4ad02e0a637189e11a3b184a6c8b46221d86"}
Jul 15 21:06:40.294067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.293610    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal" event={"ID":"72a16374957430e777a4eeb37d6601d6","Type":"ContainerStarted","Data":"7c62aa42219005e06fcdf8c8b43f0e17c5ba85fcab30d8e049208c3a66fc2b96"}
Jul 15 21:06:40.304221 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.304196    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnpcr" event={"ID":"ab1f5324-3152-4e5c-b9d3-57007a007d8c","Type":"ContainerStarted","Data":"9c01e05ab7d5ae3ba0e87eca5d0e84e3d00e582bb1ca4599f4e9e0b897d47352"}
Jul 15 21:06:40.317567 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.317536    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qms62" event={"ID":"16207249-d9af-4610-a473-fc018f680361","Type":"ContainerStarted","Data":"6fe8ebac29a44f5b3583b8df5d0943266de7b35358a8496fa866fba7e5693fe0"}
Jul 15 21:06:40.326870 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.326816    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-44.ec2.internal" podStartSLOduration=3.326799502 podStartE2EDuration="3.326799502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:40.326275531 +0000 UTC m=+4.658416623" watchObservedRunningTime="2026-07-15 21:06:40.326799502 +0000 UTC m=+4.658940595"
Jul 15 21:06:40.331298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:40.331275    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jsw54" event={"ID":"200938e3-b583-4a3c-9bdb-fcbc25af1105","Type":"ContainerStarted","Data":"18642ccfcab3541d7cca2a536c518638aeda9ebf34d3fa4d08b523c1cc537014"}
Jul 15 21:06:41.344249 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:41.344145    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal" event={"ID":"d59bb19fbb9fb26344a0b82fdd45c82a","Type":"ContainerStarted","Data":"03dcadb92b7b645f21122e806cbadd1d0d311ace4b86765f3b5501e119545fa5"}
Jul 15 21:06:41.362310 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:41.361996    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-44.ec2.internal" podStartSLOduration=4.361976092 podStartE2EDuration="4.361976092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:41.361057854 +0000 UTC m=+5.693198969" watchObservedRunningTime="2026-07-15 21:06:41.361976092 +0000 UTC m=+5.694117182"
Jul 15 21:06:41.784246 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:41.784162    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:41.784246 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:41.784212    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:41.784477 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:41.784274    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:41.784477 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.784393    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:41.784477 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.784449    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:45.784431929 +0000 UTC m=+10.116573011 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:41.785045 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.784879    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:06:41.785045 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.784901    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:06:41.785045 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.784913    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:41.785045 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.784971    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:06:45.784942375 +0000 UTC m=+10.117083459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:41.785045 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.785031    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:41.785343 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:41.785062    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:45.785052103 +0000 UTC m=+10.117193172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:42.211569 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:42.211495    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:42.211727 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:42.211651    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:42.212068 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:42.212048    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:42.212173 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:42.212143    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:42.212260 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:42.212246    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:42.212367 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:42.212350    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:44.211455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:44.211377    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:44.211455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:44.211419    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:44.211936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:44.211386    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:44.211936 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:44.211513    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:44.211936 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:44.211599    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:44.211936 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:44.211661    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:45.818534    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:45.818593    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:45.818624    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818685    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818722    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818751    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:53.818732345 +0000 UTC m=+18.150873434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818752    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818768    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:06:53.818760969 +0000 UTC m=+18.150902041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:45.818764 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818777    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:06:45.819661 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818790    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:45.819661 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:45.818836    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:06:53.818821419 +0000 UTC m=+18.150962494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:46.212723 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:46.212541    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:46.212723 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:46.212673    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:46.212927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:46.212768    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:46.212927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:46.212870    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:46.212927 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:46.212911    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:46.213087 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:46.212925    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:48.212008 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:48.211816    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:48.212008 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:48.211919    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:48.212491 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:48.212046    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:48.212491 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:48.212067    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:48.212491 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:48.212141    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:48.212491 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:48.212230    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:50.211837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:50.211805    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:50.211837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:50.211826    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:50.212312 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:50.211863    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:50.212312 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:50.211977    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:50.212404 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:50.212378    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:50.212487 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:50.212463    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:52.211534 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:52.211484    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:52.212031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:52.211617    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:52.212031 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:52.211627    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:52.212031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:52.211653    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:52.212031 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:52.211762    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:52.212031 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:52.211848    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:53.883359 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:53.883331    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:53.883370    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:53.883387    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883473    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883485    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883504    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883523    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883535    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883535    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:09.883520494 +0000 UTC m=+34.215661562 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883575    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:09.883564214 +0000 UTC m=+34.215705281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:06:53.883760 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:53.883591    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:07:09.883581225 +0000 UTC m=+34.215722294 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:06:54.211863 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:54.211792    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:54.212027 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:54.211793    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:54.212027 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:54.211912    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:54.212027 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:54.211798    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:54.212027 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:54.212014    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:54.212241 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:54.212043    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:56.212598 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:56.212562    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:56.213141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:56.212664    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:56.213141 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:56.212675    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:56.213141 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:56.212739    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:56.213141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:56.212766    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:56.213141 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:56.212871    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:58.211921 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.211711    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:06:58.212482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.211723    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:06:58.212482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.211723    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:06:58.212482 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:58.212069    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:06:58.212482 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:58.212144    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:06:58.212482 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:06:58.212196    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:06:58.376302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.376274    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:06:58.376692 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.376665    2555 generic.go:356] "Generic (PLEG): container finished" podID="a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4" containerID="3ac3d4fe81f3c538cdbc275f7702afd2a97ace2a72c3feb712d375aed8a82172" exitCode=1
Jul 15 21:06:58.376753 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.376734    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"fceb743c687cba8a0bed07a190b0756dea0269eb5d15c67b3b63c0939ccb0d0d"}
Jul 15 21:06:58.376841 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.376820    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"241c5396ddd8acdf3d07c8d48faaadda8b6bb80a1777a0484458515b96bda182"}
Jul 15 21:06:58.376946 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.376871    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerDied","Data":"3ac3d4fe81f3c538cdbc275f7702afd2a97ace2a72c3feb712d375aed8a82172"}
Jul 15 21:06:58.376946 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.376911    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"eb4606f77719cbc1b604536dc22efd298721d9151fcbb2de78d8b1ec44c99b4a"}
Jul 15 21:06:58.378230 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.378172    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nbgw5" event={"ID":"74396062-85c2-4f03-bfb9-48ce98050e33","Type":"ContainerStarted","Data":"c56d7f4954bb5c2505b0161675a20a18149e5e6c11a0e58e7285cb0f85cde9ff"}
Jul 15 21:06:58.379499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.379370    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnpcr" event={"ID":"ab1f5324-3152-4e5c-b9d3-57007a007d8c","Type":"ContainerStarted","Data":"aadd15dc31646c1bc24d8b3e49bd4e8177fa082d86bc29f3309469baecc61b95"}
Jul 15 21:06:58.380731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.380714    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qms62" event={"ID":"16207249-d9af-4610-a473-fc018f680361","Type":"ContainerStarted","Data":"0c334b7d5827341242f3b6be4e2326fc2c72a001a9b519f680eacc818920c460"}
Jul 15 21:06:58.382120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.382033    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jsw54" event={"ID":"200938e3-b583-4a3c-9bdb-fcbc25af1105","Type":"ContainerStarted","Data":"8dbb10bf5ceb5ff642a8b1ff5faffe96c09fe7e5884ae58cf5c58fe1b0084466"}
Jul 15 21:06:58.383835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.383817    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ebae01-6922-4cb9-a266-0bedf8811e6a" containerID="8f444a63642b27ce6e129a16a99479d1429f396610533c9c6473714686bdac92" exitCode=0
Jul 15 21:06:58.383927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.383906    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerDied","Data":"8f444a63642b27ce6e129a16a99479d1429f396610533c9c6473714686bdac92"}
Jul 15 21:06:58.385301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.385280    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk" event={"ID":"85aa708f-860f-4dcd-bcbd-93511f697a9c","Type":"ContainerStarted","Data":"37b7e7db977f4cafd08f9a7134eaf9c2381a9e305b052c2b15318f2984d6fd05"}
Jul 15 21:06:58.386675 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.386650    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9hhxl" event={"ID":"1fc5605d-bebc-4f92-923a-969cfb3e3d79","Type":"ContainerStarted","Data":"768819b761bc114f289a3e8ea44defd37c6c21f0c49c8ee029719a589de3ea7b"}
Jul 15 21:06:58.417611 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.417571    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9hhxl" podStartSLOduration=3.836132945 podStartE2EDuration="22.417558847s" totalImagesPullingTime="18.581425902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:58.416895791 +0000 UTC m=+22.749036881" watchObservedRunningTime="2026-07-15 21:06:58.417558847 +0000 UTC m=+22.749699936"
Jul 15 21:06:58.417702 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.417684    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qms62" podStartSLOduration=3.85870692 podStartE2EDuration="22.417679989s" totalImagesPullingTime="18.558973069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:58.397672712 +0000 UTC m=+22.729813802" watchObservedRunningTime="2026-07-15 21:06:58.417679989 +0000 UTC m=+22.749821078"
Jul 15 21:06:58.432772 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.432736    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mnpcr" podStartSLOduration=11.957571336 podStartE2EDuration="22.432728082s" totalImagesPullingTime="10.475156746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:58.432158824 +0000 UTC m=+22.764299936" watchObservedRunningTime="2026-07-15 21:06:58.432728082 +0000 UTC m=+22.764869217"
Jul 15 21:06:58.449263 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.449228    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nbgw5" podStartSLOduration=3.813317552 podStartE2EDuration="22.449215891s" totalImagesPullingTime="18.635898339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:58.448740728 +0000 UTC m=+22.780881817" watchObservedRunningTime="2026-07-15 21:06:58.449215891 +0000 UTC m=+22.781356980"
Jul 15 21:06:58.485277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.485243    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jsw54" podStartSLOduration=3.919248539 podStartE2EDuration="22.485230014s" totalImagesPullingTime="18.565981475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:58.484856862 +0000 UTC m=+22.816997951" watchObservedRunningTime="2026-07-15 21:06:58.485230014 +0000 UTC m=+22.817371104"
Jul 15 21:06:58.493529 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.493510    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:58.567045 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.567015    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:58.624889 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.624858    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:58.683301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:58.683226    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:59.390868 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.390780    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9f58w" event={"ID":"0378e56f-0e25-4ef6-96be-9e96ed50ef7d","Type":"ContainerStarted","Data":"e3f26e91d2d892dca574d18b113ad719344122314ecd85a9168e91bc88bd8edd"}
Jul 15 21:06:59.393885 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.393863    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:06:59.394348 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.394325    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"6fa8dd40a4cd6077dc5803edbeeceb553991ce97139400bce99952ab66cb9cc2"}
Jul 15 21:06:59.394446 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.394401    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"ee073feb55212c3a3a3ce8b1caab733aec8e7800013d8267dde9a7af45ecfb46"}
Jul 15 21:06:59.420697 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.420664    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:59.439930 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.439882    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9f58w" podStartSLOduration=4.875139658 podStartE2EDuration="23.439864519s" totalImagesPullingTime="18.564724861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:59.43939661 +0000 UTC m=+23.771537701" watchObservedRunningTime="2026-07-15 21:06:59.439864519 +0000 UTC m=+23.772005610"
Jul 15 21:06:59.668381 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.668355    2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 21:06:59.731217 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.731186    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:59.831890 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.831850    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:06:59.935979 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:06:59.935886    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:00.037422 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.037384    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:00.115546 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.115424    2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T21:06:59.66838065Z","UUID":"56015b77-7030-4ce9-b919-844a54f22bbd","Handler":null,"Name":"","Endpoint":""}
Jul 15 21:07:00.117134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.117112    2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 21:07:00.117257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.117156    2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 21:07:00.211291 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.211211    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:00.211291 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.211249    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:00.211490 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.211211    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:00.211490 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:00.211335    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:00.211490 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:00.211403    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:00.211490 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:00.211482    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:00.399421 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.399372    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk" event={"ID":"85aa708f-860f-4dcd-bcbd-93511f697a9c","Type":"ContainerStarted","Data":"9ac938783bb847c51122737994d523d66d41110ce6c388d97d605d3b6cce38d4"}
Jul 15 21:07:00.487457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.487360    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:07:00.672740 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.672707    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:00.733972 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:00.733906    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:01.402763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:01.402685    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk" event={"ID":"85aa708f-860f-4dcd-bcbd-93511f697a9c","Type":"ContainerStarted","Data":"b98c4e4186e3bd35a9d8dd609fb13078c528c5cb4b31c1ab5e296ac0be66d408"}
Jul 15 21:07:01.405527 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:01.405505    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:07:01.405941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:01.405912    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"7358f45572e25928101d9d7d3efaa862e549ea7df5ce194af33fa37f2f2e5908"}
Jul 15 21:07:01.427099 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:01.424946    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-msjpk" podStartSLOduration=4.128448171 podStartE2EDuration="25.424928415s" totalImagesPullingTime="21.296480244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:01.424193802 +0000 UTC m=+25.756334885" watchObservedRunningTime="2026-07-15 21:07:01.424928415 +0000 UTC m=+25.757069505"
Jul 15 21:07:02.211728 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.211700    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:02.212023 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.211701    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:02.212023 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:02.211820    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:02.212023 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:02.211882    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:02.212023 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.211701    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:02.212023 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:02.211991    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:02.232981 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.232938    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:02.478783 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.478713    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:02.992454 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.992426    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:07:02.993283 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:02.993258    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:07:03.374889 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.374706    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jsw54_200938e3-b583-4a3c-9bdb-fcbc25af1105/dns-node-resolver/0.log"
Jul 15 21:07:03.413466 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.413380    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:07:03.413805 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.413772    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"f611bcdf2bee381a5a10ab8532d6dfa5a403a1562f717bc6b965734b9cbc1be8"}
Jul 15 21:07:03.414061 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.414041    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:07:03.414276 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.414257    2555 scope.go:117] "RemoveContainer" containerID="3ac3d4fe81f3c538cdbc275f7702afd2a97ace2a72c3feb712d375aed8a82172"
Jul 15 21:07:03.415938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.415914    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerStarted","Data":"232e21d311e0a4c00947483c55460d8fb6c7ff8e8648f13bd948f2c378a7f64a"}
Jul 15 21:07:03.416881 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.416852    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mnpcr"
Jul 15 21:07:03.429766 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.429747    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:07:03.558860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:03.558834    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:04.211594 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.211564    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:04.211743 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.211570    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:04.211743 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:04.211672    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:04.211825 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:04.211762    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:04.211825 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.211571    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:04.211894 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:04.211871    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:04.377560 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.377521    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qms62_16207249-d9af-4610-a473-fc018f680361/node-ca/0.log"
Jul 15 21:07:04.419768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.419688    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ebae01-6922-4cb9-a266-0bedf8811e6a" containerID="232e21d311e0a4c00947483c55460d8fb6c7ff8e8648f13bd948f2c378a7f64a" exitCode=0
Jul 15 21:07:04.419768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.419728    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerDied","Data":"232e21d311e0a4c00947483c55460d8fb6c7ff8e8648f13bd948f2c378a7f64a"}
Jul 15 21:07:04.424012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.423993    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:07:04.424463 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.424436    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" event={"ID":"a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4","Type":"ContainerStarted","Data":"d6816a269da801463db63cea94bdacf00fdc03bfb002f5daee8d1860d90594fa"}
Jul 15 21:07:04.424683 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.424652    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:07:04.424901 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.424883    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:07:04.441364 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.441341    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:07:04.480338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.480281    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m" podStartSLOduration=9.846460198 podStartE2EDuration="28.480259618s" totalImagesPullingTime="18.63379942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:04.478513449 +0000 UTC m=+28.810654541" watchObservedRunningTime="2026-07-15 21:07:04.480259618 +0000 UTC m=+28.812400712"
Jul 15 21:07:04.567484 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.567444    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:04.622137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:04.622103    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:05.433679 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.433648    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wgwt4"]
Jul 15 21:07:05.434091 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.433748    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:05.434091 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:05.433866    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:05.436324 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.436268    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zccbq"]
Jul 15 21:07:05.436479 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.436354    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:05.436479 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:05.436448    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:05.436894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.436875    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-htj74"]
Jul 15 21:07:05.437002 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.436973    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:05.437092 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:05.437075    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:05.958599 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:05.958525    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:06.043537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:06.043512    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:06.101454 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:06.101429    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:06.431774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:06.431574    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ebae01-6922-4cb9-a266-0bedf8811e6a" containerID="f95c2064b2c0100b9b4d1e2baa8d6631ffee81c1beb0f3fe74baaecca14e5cc0" exitCode=0
Jul 15 21:07:06.431774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:06.431654    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerDied","Data":"f95c2064b2c0100b9b4d1e2baa8d6631ffee81c1beb0f3fe74baaecca14e5cc0"}
Jul 15 21:07:06.784878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:06.784793    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:06.886676 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:06.886644    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:07.211763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:07.211731    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:07.211763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:07.211767    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:07.211763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:07.211766    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:07.212214 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:07.211846    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:07.212214 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:07.211989    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:07.212214 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:07.212066    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:08.438736 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:08.438705    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ebae01-6922-4cb9-a266-0bedf8811e6a" containerID="bc2f996861c952a1a1d2420b94723eaf17ecc28e187a9742899c19720a07f86f" exitCode=0
Jul 15 21:07:08.439249 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:08.438760    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerDied","Data":"bc2f996861c952a1a1d2420b94723eaf17ecc28e187a9742899c19720a07f86f"}
Jul 15 21:07:09.211943 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.211905    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:09.212118 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.211905    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:09.212118 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.212030    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:09.212118 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.212080    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:09.212118 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.211908    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:09.212327 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.212181    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:09.558048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.557947    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:09.623332 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.623299    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:09.910537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.910450    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:09.910537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.910485    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:09.910549    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910610    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910636    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910649    2555 projected.go:196] Error preparing data for projected volume kube-api-access-wtc92 for pod openshift-network-diagnostics/network-check-target-wgwt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910704    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92 podName:566a5e0c-331e-452a-bb69-7d9502016bdf nodeName:}" failed. No retries permitted until 2026-07-15 21:07:41.910688176 +0000 UTC m=+66.242829244 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtc92" (UniqueName: "kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92") pod "network-check-target-wgwt4" (UID: "566a5e0c-331e-452a-bb69-7d9502016bdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910728    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:07:09.910771 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910768    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:07:09.911169 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910789    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret podName:543fcfc2-8dba-49bd-a5f4-5a9064532860 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:41.910773517 +0000 UTC m=+66.242914599 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret") pod "global-pull-secret-syncer-zccbq" (UID: "543fcfc2-8dba-49bd-a5f4-5a9064532860") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:07:09.911169 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:09.910814    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs podName:0fbc5aea-e9af-46f5-a603-0d46d6db44a8 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:41.910799561 +0000 UTC m=+66.242940651 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs") pod "network-metrics-daemon-htj74" (UID: "0fbc5aea-e9af-46f5-a603-0d46d6db44a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:07:10.002298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.002263    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:10.129722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.129691    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:10.258040 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.257944    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:10.341172 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.341139    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:10.423717 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.423670    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:10.527475 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.527398    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:10.875739 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:10.875631    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:11.211185 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.211107    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:11.211353 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.211107    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:11.211353 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.211249    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zccbq" podUID="543fcfc2-8dba-49bd-a5f4-5a9064532860"
Jul 15 21:07:11.211353 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.211115    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:11.211353 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.211342    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htj74" podUID="0fbc5aea-e9af-46f5-a603-0d46d6db44a8"
Jul 15 21:07:11.211532 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.211384    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wgwt4" podUID="566a5e0c-331e-452a-bb69-7d9502016bdf"
Jul 15 21:07:11.422020 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.421987    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-44.ec2.internal" event="NodeReady"
Jul 15 21:07:11.422184 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.422138    2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 21:07:11.471886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.471817    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b99lb"]
Jul 15 21:07:11.475455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.474946    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.475455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.475186    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n2fr5"]
Jul 15 21:07:11.477197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.477175    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.479272 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.479190    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 21:07:11.479626 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.479606    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 21:07:11.480350 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.479746    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 21:07:11.480350 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.479574    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g2hd4\""
Jul 15 21:07:11.481185 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.481128    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 21:07:11.481387 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.481290    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 21:07:11.481866 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.481574    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zr29g\""
Jul 15 21:07:11.481866 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.481678    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 21:07:11.481866 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.481759    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 21:07:11.484580 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.484558    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b99lb"]
Jul 15 21:07:11.491071 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.491049    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n2fr5"]
Jul 15 21:07:11.494973 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.494940    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g5zkb"]
Jul 15 21:07:11.496914 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.496897    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.499731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.499714    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 21:07:11.499996 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.499978    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mhzfs\""
Jul 15 21:07:11.500087 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.500052    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 21:07:11.508779 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.508757    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g5zkb"]
Jul 15 21:07:11.624252 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624215    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvzxv\" (UniqueName: \"kubernetes.io/projected/983aecbc-01b0-44df-b21e-edb44aeab3a2-kube-api-access-qvzxv\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.624447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624263    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-crio-socket\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.624447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624298    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.624447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624323    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/983aecbc-01b0-44df-b21e-edb44aeab3a2-config-volume\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.624447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624397    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efca4669-f0d9-4457-9d83-558551a8c66f-cert\") pod \"ingress-canary-b99lb\" (UID: \"efca4669-f0d9-4457-9d83-558551a8c66f\") " pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.624447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624423    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.624695 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624486    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fbp8\" (UniqueName: \"kubernetes.io/projected/efca4669-f0d9-4457-9d83-558551a8c66f-kube-api-access-9fbp8\") pod \"ingress-canary-b99lb\" (UID: \"efca4669-f0d9-4457-9d83-558551a8c66f\") " pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.624695 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624531    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-data-volume\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.624695 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624584    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.624695 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624614    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983aecbc-01b0-44df-b21e-edb44aeab3a2-tmp-dir\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.624695 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.624663    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cr9s\" (UniqueName: \"kubernetes.io/projected/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-kube-api-access-7cr9s\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.632369 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.632344    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:11.688198 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.688168    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:11.725875 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.725788    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvzxv\" (UniqueName: \"kubernetes.io/projected/983aecbc-01b0-44df-b21e-edb44aeab3a2-kube-api-access-qvzxv\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.725875 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.725830    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-crio-socket\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.725875 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.725858    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.725887    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/983aecbc-01b0-44df-b21e-edb44aeab3a2-config-volume\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.725940    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efca4669-f0d9-4457-9d83-558551a8c66f-cert\") pod \"ingress-canary-b99lb\" (UID: \"efca4669-f0d9-4457-9d83-558551a8c66f\") " pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.725978    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726039    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fbp8\" (UniqueName: \"kubernetes.io/projected/efca4669-f0d9-4457-9d83-558551a8c66f-kube-api-access-9fbp8\") pod \"ingress-canary-b99lb\" (UID: \"efca4669-f0d9-4457-9d83-558551a8c66f\") " pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726062    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-data-volume\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726084    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-crio-socket\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726099    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726473 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726147    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983aecbc-01b0-44df-b21e-edb44aeab3a2-tmp-dir\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.726473 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.726170    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:07:11.726473 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726182    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cr9s\" (UniqueName: \"kubernetes.io/projected/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-kube-api-access-7cr9s\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726473 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.726252    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls podName:983aecbc-01b0-44df-b21e-edb44aeab3a2 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:12.226228983 +0000 UTC m=+36.558370063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls") pod "dns-default-g5zkb" (UID: "983aecbc-01b0-44df-b21e-edb44aeab3a2") : secret "dns-default-metrics-tls" not found
Jul 15 21:07:11.726473 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.726439    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:11.726473 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726465    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-data-volume\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726744 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:11.726488    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls podName:4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:12.226473754 +0000 UTC m=+36.558614822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2fr5" (UID: "4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:11.726744 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726542    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983aecbc-01b0-44df-b21e-edb44aeab3a2-tmp-dir\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.726744 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726547    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.726744 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.726576    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/983aecbc-01b0-44df-b21e-edb44aeab3a2-config-volume\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.730580 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.730559    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efca4669-f0d9-4457-9d83-558551a8c66f-cert\") pod \"ingress-canary-b99lb\" (UID: \"efca4669-f0d9-4457-9d83-558551a8c66f\") " pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.737668 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.737645    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cr9s\" (UniqueName: \"kubernetes.io/projected/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-kube-api-access-7cr9s\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:11.737842 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.737818    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvzxv\" (UniqueName: \"kubernetes.io/projected/983aecbc-01b0-44df-b21e-edb44aeab3a2-kube-api-access-qvzxv\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:11.738272 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.738252    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fbp8\" (UniqueName: \"kubernetes.io/projected/efca4669-f0d9-4457-9d83-558551a8c66f-kube-api-access-9fbp8\") pod \"ingress-canary-b99lb\" (UID: \"efca4669-f0d9-4457-9d83-558551a8c66f\") " pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.789623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.789592    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b99lb"
Jul 15 21:07:11.867396 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.867364    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-94ldh"]
Jul 15 21:07:11.869494 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.869475    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:11.872128 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.872103    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 15 21:07:11.872241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.872145    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5b47d\""
Jul 15 21:07:11.872241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.872159    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 15 21:07:11.872241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.872197    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 15 21:07:11.872421 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.872402    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 15 21:07:11.877938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.877925    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-94ldh"]
Jul 15 21:07:11.900314 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:11.900283    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:12.029580 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.029491    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfjx2\" (UniqueName: \"kubernetes.io/projected/2fa1b2e7-c10c-4140-9441-1f91eb378214-kube-api-access-jfjx2\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.029748 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.029669    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fa1b2e7-c10c-4140-9441-1f91eb378214-signing-cabundle\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.029748 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.029705    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fa1b2e7-c10c-4140-9441-1f91eb378214-signing-key\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.109156 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.109120    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:12.130321 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.130291    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fa1b2e7-c10c-4140-9441-1f91eb378214-signing-cabundle\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.130458 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.130330    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fa1b2e7-c10c-4140-9441-1f91eb378214-signing-key\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.130458 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.130390    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfjx2\" (UniqueName: \"kubernetes.io/projected/2fa1b2e7-c10c-4140-9441-1f91eb378214-kube-api-access-jfjx2\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.131046 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.131025    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fa1b2e7-c10c-4140-9441-1f91eb378214-signing-cabundle\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.132941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.132918    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fa1b2e7-c10c-4140-9441-1f91eb378214-signing-key\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.139665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.139643    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfjx2\" (UniqueName: \"kubernetes.io/projected/2fa1b2e7-c10c-4140-9441-1f91eb378214-kube-api-access-jfjx2\") pod \"service-ca-79f68b9f4f-94ldh\" (UID: \"2fa1b2e7-c10c-4140-9441-1f91eb378214\") " pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.181554 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.181530    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh"
Jul 15 21:07:12.231467 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.231442    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:12.231592 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.231511    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:12.231641 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:12.231614    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:12.231686 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:12.231674    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls podName:4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:13.231653844 +0000 UTC m=+37.563794917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2fr5" (UID: "4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:12.231752 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:12.231615    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:07:12.231802 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:12.231765    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls podName:983aecbc-01b0-44df-b21e-edb44aeab3a2 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:13.23174769 +0000 UTC m=+37.563888766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls") pod "dns-default-g5zkb" (UID: "983aecbc-01b0-44df-b21e-edb44aeab3a2") : secret "dns-default-metrics-tls" not found
Jul 15 21:07:12.426670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.426535    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b99lb"]
Jul 15 21:07:12.429498 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.429479    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-94ldh"]
Jul 15 21:07:12.430346 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:12.430324    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefca4669_f0d9_4457_9d83_558551a8c66f.slice/crio-e5ea9b40b48e495996269e8ecd586ca63bd2dfbbad522ae147b77052a372d1ee WatchSource:0}: Error finding container e5ea9b40b48e495996269e8ecd586ca63bd2dfbbad522ae147b77052a372d1ee: Status 404 returned error can't find the container with id e5ea9b40b48e495996269e8ecd586ca63bd2dfbbad522ae147b77052a372d1ee
Jul 15 21:07:12.431578 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:12.431555    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa1b2e7_c10c_4140_9441_1f91eb378214.slice/crio-fb6f2d2721c7dfb6a9b807eed8773ff814683a7d38150ba6221268c169606cb6 WatchSource:0}: Error finding container fb6f2d2721c7dfb6a9b807eed8773ff814683a7d38150ba6221268c169606cb6: Status 404 returned error can't find the container with id fb6f2d2721c7dfb6a9b807eed8773ff814683a7d38150ba6221268c169606cb6
Jul 15 21:07:12.450707 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.450681    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerStarted","Data":"18f224b0e1417cf50e09cac66d3c302ebb14d05b4e2db7e94466f65051ca2e92"}
Jul 15 21:07:12.451986 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.451946    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh" event={"ID":"2fa1b2e7-c10c-4140-9441-1f91eb378214","Type":"ContainerStarted","Data":"fb6f2d2721c7dfb6a9b807eed8773ff814683a7d38150ba6221268c169606cb6"}
Jul 15 21:07:12.453314 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.453287    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b99lb" event={"ID":"efca4669-f0d9-4457-9d83-558551a8c66f","Type":"ContainerStarted","Data":"e5ea9b40b48e495996269e8ecd586ca63bd2dfbbad522ae147b77052a372d1ee"}
Jul 15 21:07:12.745106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.745018    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:12.965377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:12.965342    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:13.022597 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.022524    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:13.211550 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.211449    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:13.211550 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.211489    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:13.212048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.211449    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:13.214873 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.214848    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 21:07:13.215009 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.214879    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jw9wn\""
Jul 15 21:07:13.215860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.215838    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v4wmg\""
Jul 15 21:07:13.215974 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.215867    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 21:07:13.216054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.216020    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 21:07:13.216054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.216032    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 21:07:13.239244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.239221    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:13.239341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.239290    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:13.239401 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:13.239360    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:07:13.239401 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:13.239376    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:13.239499 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:13.239424    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls podName:983aecbc-01b0-44df-b21e-edb44aeab3a2 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:15.239404709 +0000 UTC m=+39.571545791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls") pod "dns-default-g5zkb" (UID: "983aecbc-01b0-44df-b21e-edb44aeab3a2") : secret "dns-default-metrics-tls" not found
Jul 15 21:07:13.239499 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:13.239443    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls podName:4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:15.239433366 +0000 UTC m=+39.571574438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2fr5" (UID: "4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:13.458670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.458598    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ebae01-6922-4cb9-a266-0bedf8811e6a" containerID="18f224b0e1417cf50e09cac66d3c302ebb14d05b4e2db7e94466f65051ca2e92" exitCode=0
Jul 15 21:07:13.458670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:13.458660    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerDied","Data":"18f224b0e1417cf50e09cac66d3c302ebb14d05b4e2db7e94466f65051ca2e92"}
Jul 15 21:07:14.467742 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:14.467704    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1ebae01-6922-4cb9-a266-0bedf8811e6a" containerID="f93474243a1a9c1308cc960a14b1c3e8657dbf0f7feab7705948a3c3e1caed3b" exitCode=0
Jul 15 21:07:14.468180 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:14.467750    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerDied","Data":"f93474243a1a9c1308cc960a14b1c3e8657dbf0f7feab7705948a3c3e1caed3b"}
Jul 15 21:07:15.256192 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.256168    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:15.256274 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.256225    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:15.256326 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:15.256309    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:15.256384 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:15.256374    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls podName:4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:19.25635858 +0000 UTC m=+43.588499648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2fr5" (UID: "4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:15.256439 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:15.256317    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:07:15.256475 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:15.256438    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls podName:983aecbc-01b0-44df-b21e-edb44aeab3a2 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:19.256423674 +0000 UTC m=+43.588564743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls") pod "dns-default-g5zkb" (UID: "983aecbc-01b0-44df-b21e-edb44aeab3a2") : secret "dns-default-metrics-tls" not found
Jul 15 21:07:15.475459 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.475216    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" event={"ID":"b1ebae01-6922-4cb9-a266-0bedf8811e6a","Type":"ContainerStarted","Data":"541e9d1246d47bd8fbd732283acf3a192573af643ddb189eb83a6c8d92206f68"}
Jul 15 21:07:15.476917 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.476886    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh" event={"ID":"2fa1b2e7-c10c-4140-9441-1f91eb378214","Type":"ContainerStarted","Data":"76aff9b78c11925c5a29b6cbeed5d7e14da14537f939079f3f22a2d35c348809"}
Jul 15 21:07:15.478302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.478279    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b99lb" event={"ID":"efca4669-f0d9-4457-9d83-558551a8c66f","Type":"ContainerStarted","Data":"8302772240cb044231f7c22866ac0b129c5226b2c80970d8c5c3313140772a8f"}
Jul 15 21:07:15.503473 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.503370    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vjmbd" podStartSLOduration=10.248749543 podStartE2EDuration="39.503353456s" totalImagesPullingTime="29.254603913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:15.501499896 +0000 UTC m=+39.833640986" watchObservedRunningTime="2026-07-15 21:07:15.503353456 +0000 UTC m=+39.835494546"
Jul 15 21:07:15.520126 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.520077    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-94ldh" podStartSLOduration=1.6759108889999998 podStartE2EDuration="4.52006133s" totalImagesPullingTime="2.844150441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:15.518994955 +0000 UTC m=+39.851136059" watchObservedRunningTime="2026-07-15 21:07:15.52006133 +0000 UTC m=+39.852202416"
Jul 15 21:07:15.537341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.537303    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b99lb" podStartSLOduration=1.68421653 podStartE2EDuration="4.537288745s" totalImagesPullingTime="2.853072215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:15.536891613 +0000 UTC m=+39.869032725" watchObservedRunningTime="2026-07-15 21:07:15.537288745 +0000 UTC m=+39.869429834"
Jul 15 21:07:15.687783 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:15.687751    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:16.407137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:16.407101    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:16.470839 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:16.470805    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:16.785331 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:16.785244    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:16.940612 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:16.940573    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:17.051879 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:17.051779    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:17.154827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:17.154794    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:17.334246 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:17.334145    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:17.421823 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:17.421791    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:17.526203 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:17.526155    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:18.679931 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:18.679896    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:18.889210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:18.889156    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:19.019404 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.019321    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:19.120250 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.120219    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:19.285539 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.285456    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:19.288540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.288517    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:19.288690 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.288579    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:19.288690 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:19.288687    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:19.288792 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:19.288692    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:07:19.288792 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:19.288752    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls podName:4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:27.28873502 +0000 UTC m=+51.620876090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n2fr5" (UID: "4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:07:19.288792 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:07:19.288770    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls podName:983aecbc-01b0-44df-b21e-edb44aeab3a2 nodeName:}" failed. No retries permitted until 2026-07-15 21:07:27.288761404 +0000 UTC m=+51.620902472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls") pod "dns-default-g5zkb" (UID: "983aecbc-01b0-44df-b21e-edb44aeab3a2") : secret "dns-default-metrics-tls" not found
Jul 15 21:07:19.477388 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.477355    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:19.578470 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.578387    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:19.633905 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:19.633876    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:20.083388 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:20.083357    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:20.145254 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:20.145227    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:20.477463 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:20.477432    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:20.883082 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:20.883038    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:20.939657 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:20.939625    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:21.279112 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:21.279033    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:21.484520 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:21.484490    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:21.680556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:21.680523    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:22.085499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:22.085425    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:22.481635 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:22.481608    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:23.079077 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:23.079039    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:23.161305 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:23.161278    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:23.679872 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:23.679837    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:23.882897 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:23.882860    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:24.279937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:24.279894    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:24.480496 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:24.480463    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:24.881663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:24.881630    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:25.479685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:25.479655    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:26.078659 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:26.078634    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:26.300770 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:26.300737    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:26.492797 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:26.492768    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:26.680025 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:26.679995    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:27.285801 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.285768    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:27.341371 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.341345    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:27.341524 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.341396    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:27.345129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.345106    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/983aecbc-01b0-44df-b21e-edb44aeab3a2-metrics-tls\") pod \"dns-default-g5zkb\" (UID: \"983aecbc-01b0-44df-b21e-edb44aeab3a2\") " pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:27.345225 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.345110    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n2fr5\" (UID: \"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5\") " pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:27.396067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.396042    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n2fr5"
Jul 15 21:07:27.406211 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.406194    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:27.494661 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.494621    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:27.558280 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.558254    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n2fr5"]
Jul 15 21:07:27.560532 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:27.560507    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8880c8_d47f_4a2e_bcd1_0f5b9c63d5f5.slice/crio-415987e7f8ade6eeb4030468685e63f019f23cc0f63b4da84de882d24ddb2059 WatchSource:0}: Error finding container 415987e7f8ade6eeb4030468685e63f019f23cc0f63b4da84de882d24ddb2059: Status 404 returned error can't find the container with id 415987e7f8ade6eeb4030468685e63f019f23cc0f63b4da84de882d24ddb2059
Jul 15 21:07:27.572757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:27.572736    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g5zkb"]
Jul 15 21:07:27.575580 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:27.575555    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983aecbc_01b0_44df_b21e_edb44aeab3a2.slice/crio-277c1f6f45c1374e993d70a35610daaf008c8f884fe3d179f14ae04e49da8294 WatchSource:0}: Error finding container 277c1f6f45c1374e993d70a35610daaf008c8f884fe3d179f14ae04e49da8294: Status 404 returned error can't find the container with id 277c1f6f45c1374e993d70a35610daaf008c8f884fe3d179f14ae04e49da8294
Jul 15 21:07:28.512154 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:28.512114    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g5zkb" event={"ID":"983aecbc-01b0-44df-b21e-edb44aeab3a2","Type":"ContainerStarted","Data":"277c1f6f45c1374e993d70a35610daaf008c8f884fe3d179f14ae04e49da8294"}
Jul 15 21:07:28.514024 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:28.513990    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2fr5" event={"ID":"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5","Type":"ContainerStarted","Data":"415987e7f8ade6eeb4030468685e63f019f23cc0f63b4da84de882d24ddb2059"}
Jul 15 21:07:30.524141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:30.524109    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g5zkb" event={"ID":"983aecbc-01b0-44df-b21e-edb44aeab3a2","Type":"ContainerStarted","Data":"2714bb7ab36e9d5aa7c7fda111929566bb918ee84bf00d90ed6baff9187ef80a"}
Jul 15 21:07:31.443752 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.443718    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"]
Jul 15 21:07:31.461386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.461350    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"]
Jul 15 21:07:31.461543 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.461507    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.464370 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.464342    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.474731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474668    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.474731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474713    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4857d523-c49c-48fa-812c-eecaa6b246f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bdfd95f46-kzw74\" (UID: \"4857d523-c49c-48fa-812c-eecaa6b246f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.474915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474748    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkjbz\" (UniqueName: \"kubernetes.io/projected/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-kube-api-access-wkjbz\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.474915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474801    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-ca\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.474915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474855    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-hub\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.474915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474881    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.474915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474913    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c87dx\" (UniqueName: \"kubernetes.io/projected/4857d523-c49c-48fa-812c-eecaa6b246f4-kube-api-access-c87dx\") pod \"managed-serviceaccount-addon-agent-6bdfd95f46-kzw74\" (UID: \"4857d523-c49c-48fa-812c-eecaa6b246f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.475190 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.474941    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.479736 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.479687    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"]
Jul 15 21:07:31.484424 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.484364    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"]
Jul 15 21:07:31.484550 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.484468    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 21:07:31.484611 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.484577    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2m9mz\""
Jul 15 21:07:31.484669 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.484471    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 21:07:31.485121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.484896    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 15 21:07:31.485121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.484914    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 21:07:31.485288 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.485270    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 15 21:07:31.485714 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.485694    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 15 21:07:31.486206 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.486029    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 15 21:07:31.486744 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.486444    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 21:07:31.575512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575479    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-ca\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.575512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575519    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-hub\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.576080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575543    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.576080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575573    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c87dx\" (UniqueName: \"kubernetes.io/projected/4857d523-c49c-48fa-812c-eecaa6b246f4-kube-api-access-c87dx\") pod \"managed-serviceaccount-addon-agent-6bdfd95f46-kzw74\" (UID: \"4857d523-c49c-48fa-812c-eecaa6b246f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.576080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575602    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.576080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575682    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.576080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575713    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4857d523-c49c-48fa-812c-eecaa6b246f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bdfd95f46-kzw74\" (UID: \"4857d523-c49c-48fa-812c-eecaa6b246f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.576080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.575738    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkjbz\" (UniqueName: \"kubernetes.io/projected/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-kube-api-access-wkjbz\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.576451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.576426    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.578671 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.578646    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.578808 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.578763    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-ca\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.579019 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.578997    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4857d523-c49c-48fa-812c-eecaa6b246f4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bdfd95f46-kzw74\" (UID: \"4857d523-c49c-48fa-812c-eecaa6b246f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.579265 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.579242    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-hub\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.579639 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.579612    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.592806 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.592781    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkjbz\" (UniqueName: \"kubernetes.io/projected/3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40-kube-api-access-wkjbz\") pod \"cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9\" (UID: \"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.593368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.593337    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c87dx\" (UniqueName: \"kubernetes.io/projected/4857d523-c49c-48fa-812c-eecaa6b246f4-kube-api-access-c87dx\") pod \"managed-serviceaccount-addon-agent-6bdfd95f46-kzw74\" (UID: \"4857d523-c49c-48fa-812c-eecaa6b246f4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.783857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.783780    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"
Jul 15 21:07:31.791341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.791310    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"
Jul 15 21:07:31.829591 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.829558    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:31.930207 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:31.930176    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:32.809981 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:32.809914    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74"]
Jul 15 21:07:32.813649 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:32.813606    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4857d523_c49c_48fa_812c_eecaa6b246f4.slice/crio-a0e1a3f5ad3790f5b4d1510859fb59788f8bb8e1cac9dec57359af9231931700 WatchSource:0}: Error finding container a0e1a3f5ad3790f5b4d1510859fb59788f8bb8e1cac9dec57359af9231931700: Status 404 returned error can't find the container with id a0e1a3f5ad3790f5b4d1510859fb59788f8bb8e1cac9dec57359af9231931700
Jul 15 21:07:32.830813 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:32.830796    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9"]
Jul 15 21:07:32.832828 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:32.832807    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dd6b2ef_8e7e_44ed_b2c9_ce5c012abf40.slice/crio-d6b4396db9b870db8a9289c257d36227e02bf7d48c84a559ab1af4be27d58251 WatchSource:0}: Error finding container d6b4396db9b870db8a9289c257d36227e02bf7d48c84a559ab1af4be27d58251: Status 404 returned error can't find the container with id d6b4396db9b870db8a9289c257d36227e02bf7d48c84a559ab1af4be27d58251
Jul 15 21:07:33.503829 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.503784    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:33.537932 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.537870    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74" event={"ID":"4857d523-c49c-48fa-812c-eecaa6b246f4","Type":"ContainerStarted","Data":"a0e1a3f5ad3790f5b4d1510859fb59788f8bb8e1cac9dec57359af9231931700"}
Jul 15 21:07:33.543410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.543379    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g5zkb" event={"ID":"983aecbc-01b0-44df-b21e-edb44aeab3a2","Type":"ContainerStarted","Data":"0df8de07a742c44dcdd3fbf2381f83a8300dafa1b1adb08f094404432f96ac84"}
Jul 15 21:07:33.543793 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.543745    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:33.545457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.545432    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9" event={"ID":"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40","Type":"ContainerStarted","Data":"d6b4396db9b870db8a9289c257d36227e02bf7d48c84a559ab1af4be27d58251"}
Jul 15 21:07:33.547390 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.547369    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2fr5" event={"ID":"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5","Type":"ContainerStarted","Data":"3fc01dfa8ad09c07c5e5468cc32c04442157b439545654f89317df5f8db7f9d1"}
Jul 15 21:07:33.572138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.572083    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g5zkb" podStartSLOduration=20.162532881 podStartE2EDuration="22.572065037s" totalImagesPullingTime="2.409532156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:33.571158912 +0000 UTC m=+57.903300028" watchObservedRunningTime="2026-07-15 21:07:33.572065037 +0000 UTC m=+57.904206122"
Jul 15 21:07:33.600503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.600449    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:33.767370 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.767286    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:33.890506 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:33.890478    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:34.011143 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:34.011109    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:34.083050 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:34.082906    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:34.181622 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:34.181586    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:34.604949 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:34.604919    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:34.688556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:34.688502    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:35.782658 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:35.782614    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:35.848155 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:35.848111    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:36.449759 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:36.449734    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4r92m"
Jul 15 21:07:36.630900 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:36.630823    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:36.696138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:36.696103    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:38.921163 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:38.921132    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:38.984081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:38.984040    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:39.282392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.282368    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:39.572743 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.572657    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9" event={"ID":"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40","Type":"ContainerStarted","Data":"2291df9603e4bb322ec3829e75d0750fe4bd7366250047dede6aa93dbdcf5b52"}
Jul 15 21:07:39.572743 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.572702    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9" event={"ID":"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40","Type":"ContainerStarted","Data":"587f1d9d08c3dff56105e1e47a6c9b2dce1dbe95e7f35760c1ffd1ec97967f8b"}
Jul 15 21:07:39.572743 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.572724    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9" event={"ID":"3dd6b2ef-8e7e-44ed-b2c9-ce5c012abf40","Type":"ContainerStarted","Data":"c3afe4ae64261d3897be6b3f1f0463b4669e55f7b53ad492e3b641ccde9db205"}
Jul 15 21:07:39.574430 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.574403    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2fr5" event={"ID":"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5","Type":"ContainerStarted","Data":"f3303180b14b7fdf85604e702997907b2fbfd2f6b203f8eebf912c8dfa9c8a76"}
Jul 15 21:07:39.575727 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.575698    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74" event={"ID":"4857d523-c49c-48fa-812c-eecaa6b246f4","Type":"ContainerStarted","Data":"7d67d39b74d7fe6b9cb1434aad17426e7d1baa091599c486caef5cd709dbec48"}
Jul 15 21:07:39.595064 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:39.595019    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f56cdfcb7-8p2x9" podStartSLOduration=2.169065343 podStartE2EDuration="8.595006653s" totalImagesPullingTime="6.42594131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:39.593625426 +0000 UTC m=+63.925766521" watchObservedRunningTime="2026-07-15 21:07:39.595006653 +0000 UTC m=+63.927147740"
Jul 15 21:07:40.555776 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:40.555748    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g5zkb"
Jul 15 21:07:40.575769 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:40.575713    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bdfd95f46-kzw74" podStartSLOduration=3.14994396 podStartE2EDuration="9.575692725s" totalImagesPullingTime="6.425748765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:39.612547644 +0000 UTC m=+63.944688730" watchObservedRunningTime="2026-07-15 21:07:40.575692725 +0000 UTC m=+64.907833816"
Jul 15 21:07:40.805561 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:40.805522    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:40.865727 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:40.865639    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:41.964164 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.964121    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:41.964658 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.964198    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:41.964658 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.964305    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:41.967341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.967321    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 21:07:41.968547 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.968513    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 21:07:41.968547 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.968529    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 21:07:41.977464 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.977440    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fbc5aea-e9af-46f5-a603-0d46d6db44a8-metrics-certs\") pod \"network-metrics-daemon-htj74\" (UID: \"0fbc5aea-e9af-46f5-a603-0d46d6db44a8\") " pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:41.977673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.977644    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/543fcfc2-8dba-49bd-a5f4-5a9064532860-original-pull-secret\") pod \"global-pull-secret-syncer-zccbq\" (UID: \"543fcfc2-8dba-49bd-a5f4-5a9064532860\") " pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:41.977792 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.977770    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 21:07:41.987910 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:41.987883    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtc92\" (UniqueName: \"kubernetes.io/projected/566a5e0c-331e-452a-bb69-7d9502016bdf-kube-api-access-wtc92\") pod \"network-check-target-wgwt4\" (UID: \"566a5e0c-331e-452a-bb69-7d9502016bdf\") " pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:42.024887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:42.024839    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zccbq"
Jul 15 21:07:42.035899 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:42.035874    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v4wmg\""
Jul 15 21:07:42.041894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:42.041864    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jw9wn\""
Jul 15 21:07:42.043982 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:42.043948    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:42.049632 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:42.049609    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htj74"
Jul 15 21:07:43.031356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.031335    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wgwt4"]
Jul 15 21:07:43.033881 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:43.033854    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566a5e0c_331e_452a_bb69_7d9502016bdf.slice/crio-78012730bc82f45963dad689d4755d9f6007383b5675ec0d5234b585b45d4af2 WatchSource:0}: Error finding container 78012730bc82f45963dad689d4755d9f6007383b5675ec0d5234b585b45d4af2: Status 404 returned error can't find the container with id 78012730bc82f45963dad689d4755d9f6007383b5675ec0d5234b585b45d4af2
Jul 15 21:07:43.251448 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.251426    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zccbq"]
Jul 15 21:07:43.253733 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.253711    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-htj74"]
Jul 15 21:07:43.254179 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:43.254156    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543fcfc2_8dba_49bd_a5f4_5a9064532860.slice/crio-3f6d10fe9bba702caa53505a13bda453da611a61459676489f190ce5143c1006 WatchSource:0}: Error finding container 3f6d10fe9bba702caa53505a13bda453da611a61459676489f190ce5143c1006: Status 404 returned error can't find the container with id 3f6d10fe9bba702caa53505a13bda453da611a61459676489f190ce5143c1006
Jul 15 21:07:43.256743 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:43.256712    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbc5aea_e9af_46f5_a603_0d46d6db44a8.slice/crio-2209087433bf77671e9c2ecc1e29de1229b5a29b9ee28347e465ec9b2a62f44b WatchSource:0}: Error finding container 2209087433bf77671e9c2ecc1e29de1229b5a29b9ee28347e465ec9b2a62f44b: Status 404 returned error can't find the container with id 2209087433bf77671e9c2ecc1e29de1229b5a29b9ee28347e465ec9b2a62f44b
Jul 15 21:07:43.591460 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.591376    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n2fr5" event={"ID":"4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5","Type":"ContainerStarted","Data":"f5b7165296f9462fe36824a0259bb626e51e1e46f129997a3e2d389dbeaa244e"}
Jul 15 21:07:43.592724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.592692    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wgwt4" event={"ID":"566a5e0c-331e-452a-bb69-7d9502016bdf","Type":"ContainerStarted","Data":"78012730bc82f45963dad689d4755d9f6007383b5675ec0d5234b585b45d4af2"}
Jul 15 21:07:43.594094 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.594066    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zccbq" event={"ID":"543fcfc2-8dba-49bd-a5f4-5a9064532860","Type":"ContainerStarted","Data":"3f6d10fe9bba702caa53505a13bda453da611a61459676489f190ce5143c1006"}
Jul 15 21:07:43.595205 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.595180    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htj74" event={"ID":"0fbc5aea-e9af-46f5-a603-0d46d6db44a8","Type":"ContainerStarted","Data":"2209087433bf77671e9c2ecc1e29de1229b5a29b9ee28347e465ec9b2a62f44b"}
Jul 15 21:07:43.611790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:43.611738    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n2fr5" podStartSLOduration=17.052777874 podStartE2EDuration="32.611724308s" totalImagesPullingTime="15.558946434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:07:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:43.61054682 +0000 UTC m=+67.942687949" watchObservedRunningTime="2026-07-15 21:07:43.611724308 +0000 UTC m=+67.943865398"
Jul 15 21:07:44.260672 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:44.260635    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:45.608805 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:45.608765    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htj74" event={"ID":"0fbc5aea-e9af-46f5-a603-0d46d6db44a8","Type":"ContainerStarted","Data":"92a7037dcd26d5d97d063ec31f83c64c81e30546e0b01d44bdf8206e066ff15e"}
Jul 15 21:07:46.379626 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.379596    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5rbbn"]
Jul 15 21:07:46.392836 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.392804    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.396434 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.396409    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 21:07:46.396565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.396489    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c6c44\""
Jul 15 21:07:46.396565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.396508    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 21:07:46.397612 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.397592    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 21:07:46.397786 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.397733    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 21:07:46.397860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.397827    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 21:07:46.397980 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.397940    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 21:07:46.498554 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498519    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-wtmp\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498560    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-metrics-client-ca\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498592    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-758tj\" (UniqueName: \"kubernetes.io/projected/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-kube-api-access-758tj\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498619    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498669    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498692    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-sys\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498932 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498790    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-tls\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498932 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498827    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-textfile\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.498932 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.498866    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-root\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.505515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.505484    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:46.600157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600075    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-sys\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600129    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-tls\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600158    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-textfile\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600191    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-root\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600195    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-sys\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600222    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-wtmp\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600246    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-metrics-client-ca\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600272    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-758tj\" (UniqueName: \"kubernetes.io/projected/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-kube-api-access-758tj\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600301    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600368 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600346    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600718 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600452    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-wtmp\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600718 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600521    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-textfile\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.600718 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.600523    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-root\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.601393 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.601368    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.601541 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.601396    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-metrics-client-ca\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.603261 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.603217    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.603836 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.603813    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-node-exporter-tls\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.606787 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.606760    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:46.610669 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.610623    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-758tj\" (UniqueName: \"kubernetes.io/projected/c3dbfa95-7bcf-42ea-a451-acd94369b0e7-kube-api-access-758tj\") pod \"node-exporter-5rbbn\" (UID: \"c3dbfa95-7bcf-42ea-a451-acd94369b0e7\") " pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.704685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.704655    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5rbbn"
Jul 15 21:07:46.706615 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:46.706580    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:47.342470 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:47.342440    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:47.442333 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:47.442300    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:47.795856 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:47.795826    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dbfa95_7bcf_42ea_a451_acd94369b0e7.slice/crio-77d4ed56fff50d755029b7bbd7c10d1cfbb36d5d718fb5f9fbe68c480b1637cc WatchSource:0}: Error finding container 77d4ed56fff50d755029b7bbd7c10d1cfbb36d5d718fb5f9fbe68c480b1637cc: Status 404 returned error can't find the container with id 77d4ed56fff50d755029b7bbd7c10d1cfbb36d5d718fb5f9fbe68c480b1637cc
Jul 15 21:07:47.832879 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:47.832849    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:47.929155 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:47.929119    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:48.621034 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.620994    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wgwt4" event={"ID":"566a5e0c-331e-452a-bb69-7d9502016bdf","Type":"ContainerStarted","Data":"c3dd1ee347848df90462c2573203d00107f3508e5eef685afe46eb5b807cb862"}
Jul 15 21:07:48.621241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.621102    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:07:48.622334 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.622306    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5rbbn" event={"ID":"c3dbfa95-7bcf-42ea-a451-acd94369b0e7","Type":"ContainerStarted","Data":"77d4ed56fff50d755029b7bbd7c10d1cfbb36d5d718fb5f9fbe68c480b1637cc"}
Jul 15 21:07:48.623877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.623835    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zccbq" event={"ID":"543fcfc2-8dba-49bd-a5f4-5a9064532860","Type":"ContainerStarted","Data":"090fc0b729ee94ff38d2430ed808b15eeba49d94321e2de9f883ea30553f7563"}
Jul 15 21:07:48.625632 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.625611    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htj74" event={"ID":"0fbc5aea-e9af-46f5-a603-0d46d6db44a8","Type":"ContainerStarted","Data":"4cbf3bd4e520a2adacdffc6eba93300f6f36883731c0a8a7ecf16675d657c8d6"}
Jul 15 21:07:48.641405 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.641353    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wgwt4" podStartSLOduration=67.880179613 podStartE2EDuration="1m12.641333316s" totalImagesPullingTime="4.761153703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:48.640313025 +0000 UTC m=+72.972454140" watchObservedRunningTime="2026-07-15 21:07:48.641333316 +0000 UTC m=+72.973474408"
Jul 15 21:07:48.657872 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.657831    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-htj74" podStartSLOduration=70.883824804 podStartE2EDuration="1m12.65781879s" totalImagesPullingTime="1.773993986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:48.657345869 +0000 UTC m=+72.989486958" watchObservedRunningTime="2026-07-15 21:07:48.65781879 +0000 UTC m=+72.989959880"
Jul 15 21:07:48.677878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.677826    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zccbq" podStartSLOduration=68.125804019 podStartE2EDuration="1m12.677808088s" totalImagesPullingTime="4.552004069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:48.676808902 +0000 UTC m=+73.008949993" watchObservedRunningTime="2026-07-15 21:07:48.677808088 +0000 UTC m=+73.009949179"
Jul 15 21:07:48.775149 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.775111    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:48.872300 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:48.872240    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:49.021592 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.021570    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:49.123071 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.123009    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:49.367482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.367448    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:49.465513 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.465435    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:49.631459 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.631424    2555 generic.go:356] "Generic (PLEG): container finished" podID="c3dbfa95-7bcf-42ea-a451-acd94369b0e7" containerID="cdcfb2b47e45003a18eacc65fd0d8e8eadd247f76f7b84805854e87c86fd6acc" exitCode=0
Jul 15 21:07:49.631654 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.631532    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5rbbn" event={"ID":"c3dbfa95-7bcf-42ea-a451-acd94369b0e7","Type":"ContainerDied","Data":"cdcfb2b47e45003a18eacc65fd0d8e8eadd247f76f7b84805854e87c86fd6acc"}
Jul 15 21:07:49.867303 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:49.867277    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:50.338998 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:50.338969    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:50.637170 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:50.637137    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5rbbn" event={"ID":"c3dbfa95-7bcf-42ea-a451-acd94369b0e7","Type":"ContainerStarted","Data":"dfe532469c9a568aa8030852c581dbdd672c345e1317a5eaab766f7d4d8486c8"}
Jul 15 21:07:50.637350 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:50.637193    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5rbbn" event={"ID":"c3dbfa95-7bcf-42ea-a451-acd94369b0e7","Type":"ContainerStarted","Data":"f9fbc7102f7ee7274202fb29b51d2370a63d71ec5756d908967cfd62a74a0da6"}
Jul 15 21:07:50.658545 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:50.658499    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5rbbn" podStartSLOduration=3.5274903589999997 podStartE2EDuration="4.658484808s" totalImagesPullingTime="1.067259253s" totalInitContainerRuntime="63.735196ms" isStatefulPod=true podCreationTimestamp="2026-07-15 21:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:50.656993198 +0000 UTC m=+74.989134310" watchObservedRunningTime="2026-07-15 21:07:50.658484808 +0000 UTC m=+74.990625928"
Jul 15 21:07:51.127280 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:51.127249    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:51.356822 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:51.356792    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:52.633227 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.633199    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:52.638330 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.638306    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:07:52.640672 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.640656    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.643982 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.643928    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 21:07:52.644208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.644186    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 21:07:52.647696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.645629    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 21:07:52.647696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.647320    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 21:07:52.647696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.647492    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 21:07:52.648172 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.647890    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 21:07:52.648523 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.648503    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 21:07:52.648632 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.648615    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 21:07:52.649120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.649081    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 21:07:52.649278 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.649259    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 21:07:52.649420 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.649093    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1iu1fl5j903tr\""
Jul 15 21:07:52.650405 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.650353    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 21:07:52.650405 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.650366    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 21:07:52.650558 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.650447    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 21:07:52.650558 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.650366    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 21:07:52.650665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.650601    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8w2mz\""
Jul 15 21:07:52.652160 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.652145    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 21:07:52.663702 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.663681    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:07:52.749436 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749405    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749436 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749443    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749464    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749481    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749537    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749604    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749635    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749663    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.749689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749687    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749710    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749739    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749766    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749790    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749814    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749844    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749869    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749894    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.749979    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lpfz\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-kube-api-access-5lpfz\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.750016    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.750129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.750043    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850744 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850709    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850752    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850784    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850804    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850822    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850837    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850855    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850878    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850899    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.850936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850921    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850946    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.850996    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851020    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851048    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851075    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lpfz\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-kube-api-access-5lpfz\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851100    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851128    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851154    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851181    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851202    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.851883 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.851857    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.852707 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.852674    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.854285 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.854258    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.854738 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.854613    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.854847 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.854785    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.855938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.855054    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.855938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.855057    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.855938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.855384    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.855938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.855726    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.855938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.855831    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.855938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.855936    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.856320 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.856107    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.856320 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.856166    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.856429 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.856371    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.856565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.856540    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.856677 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.856620    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.856734 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.856705    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.857062 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.857040    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.857978 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.857943    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.866193 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.866176    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lpfz\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-kube-api-access-5lpfz\") pod \"prometheus-k8s-0\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:52.952484 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:52.952392    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:07:53.088820 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:07:53.088790    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d0a641_40fb_4926_8b45_6d71aedcba5f.slice/crio-550d2b29727370191c40da903d32398059f7c8088fe478cf5d1719fd6a8dfe71 WatchSource:0}: Error finding container 550d2b29727370191c40da903d32398059f7c8088fe478cf5d1719fd6a8dfe71: Status 404 returned error can't find the container with id 550d2b29727370191c40da903d32398059f7c8088fe478cf5d1719fd6a8dfe71
Jul 15 21:07:53.089342 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:53.089320    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:07:53.658613 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:53.658580    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"550d2b29727370191c40da903d32398059f7c8088fe478cf5d1719fd6a8dfe71"}
Jul 15 21:07:53.797164 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:53.797124    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:53.853103 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:53.853071    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:54.178847 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.178813    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:54.432927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.432895    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:54.504862 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.504830    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:54.638598 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.638567    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:54.663324 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.663300    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e" exitCode=0
Jul 15 21:07:54.663681 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.663350    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"}
Jul 15 21:07:54.831060 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:54.830971    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:55.044814 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:55.044786    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:55.659073 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:55.659036    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:55.855659 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:55.855607    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:56.726402 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:56.726356    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:56.819338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:56.819304    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:07:57.676213 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:57.676178    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"}
Jul 15 21:07:57.676551 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:07:57.676272    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"}
Jul 15 21:08:00.176566 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.176535    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:00.277071 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.277041    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:00.690140 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.690103    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"}
Jul 15 21:08:00.690304 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.690172    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"}
Jul 15 21:08:00.690304 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.690199    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"}
Jul 15 21:08:00.690304 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.690228    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerStarted","Data":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"}
Jul 15 21:08:00.735769 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:00.735724    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.871148765 podStartE2EDuration="8.735709372s" totalImagesPullingTime="5.839949877s" totalInitContainerRuntime="24.61073ms" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:00.735116093 +0000 UTC m=+85.067257183" watchObservedRunningTime="2026-07-15 21:08:00.735709372 +0000 UTC m=+85.067850461"
Jul 15 21:08:02.952927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:02.952892    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:08:03.880479 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:03.880446    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:04.574537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:04.574501    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:04.638385 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:04.638352    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:05.642442 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:05.642405    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:05.743309 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:05.743271    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:06.682451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:06.682420    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:06.745084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:06.745057    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:10.501164 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:10.501129    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:10.565305 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:10.565277    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:11.874702 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:11.874676    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:11.975076 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:11.975035    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:12.075839 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:12.075807    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:15.105011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:15.104976    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:15.162110 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:15.162078    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:19.567236 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:19.567181    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:19.633976 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:19.633933    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wgwt4"
Jul 15 21:08:19.663030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:19.662996    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:19.718385 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:19.718362    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:21.703124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:21.703076    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:21.794145 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:21.794118    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:25.696446 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:25.696407    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:25.757508 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:25.757477    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:27.482947 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:27.482913    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:27.592066 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:27.592036    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:33.083247 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:33.083208    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:33.151978 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:33.151928    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:34.572747 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:34.572711    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:34.673183 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:34.673146    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:49.898941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:49.898910    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:49.959426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:49.959394    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:08:52.953265 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:52.953231    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:08:52.972504 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:52.972478    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:08:53.883079 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:53.883049    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:08:55.446971 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:08:55.446932    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:05.719841 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:05.719798    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:06.011411 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:06.011326    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:11.116425 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116391    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:09:11.116810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116711    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="prometheus" containerID="cri-o://f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995" gracePeriod=600
Jul 15 21:09:11.116810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116736    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-web" containerID="cri-o://69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82" gracePeriod=600
Jul 15 21:09:11.116810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116779    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy" containerID="cri-o://6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171" gracePeriod=600
Jul 15 21:09:11.117006 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116736    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13" gracePeriod=600
Jul 15 21:09:11.117006 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116812    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="config-reloader" containerID="cri-o://ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c" gracePeriod=600
Jul 15 21:09:11.117006 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.116749    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="thanos-sidecar" containerID="cri-o://0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d" gracePeriod=600
Jul 15 21:09:11.343437 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.343413    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:11.451077 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451050    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-tls-assets\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451095    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-0\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451202    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config-out\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451337 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451250    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451337 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451283    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-metrics-client-certs\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451337 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451316    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-metrics-client-ca\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451350    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-serving-certs-ca-bundle\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451384    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-grpc-tls\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451427    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-trusted-ca-bundle\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451464    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-2\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451494    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-thanos-prometheus-http-client-file\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451522    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-1\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451552    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-kubelet-serving-ca-bundle\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451576    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-kube-rbac-proxy\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451603    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-db\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451634    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lpfz\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-kube-api-access-5lpfz\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.451673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451671    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.452037 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451699    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-web-config\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.452037 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451723    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-tls\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.452037 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.451748    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\" (UID: \"e8d0a641-40fb-4926-8b45-6d71aedcba5f\") "
Jul 15 21:09:11.452195 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.452051    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.452663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.452453    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.452663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.452528    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.452841 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.452726    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.452841 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.452774    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.454041 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.453887    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config-out" (OuterVolumeSpecName: "config-out") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:09:11.454150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.454124    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.454298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.454275    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config" (OuterVolumeSpecName: "config") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.454435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.454401    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:09:11.454703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.454680    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:09:11.454786 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.454698    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.454849 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.454829    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:09:11.455275 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.455252    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.455784 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.455743    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.455880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.455836    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.456241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.456216    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.456542 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.456526    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-kube-api-access-5lpfz" (OuterVolumeSpecName: "kube-api-access-5lpfz") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "kube-api-access-5lpfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:09:11.456599 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.456582    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.456865 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.456850    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.464547 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.464527    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-web-config" (OuterVolumeSpecName: "web-config") pod "e8d0a641-40fb-4926-8b45-6d71aedcba5f" (UID: "e8d0a641-40fb-4926-8b45-6d71aedcba5f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:09:11.553299 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553263    2555 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553299 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553299    2555 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-metrics-client-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553299 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553309    2555 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-metrics-client-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553321    2555 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553332    2555 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-grpc-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553340    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553349    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553358    2555 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553366    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553376    2555 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553385    2555 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-kube-rbac-proxy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553394    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-db\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553401    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lpfz\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-kube-api-access-5lpfz\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553411    2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553418    2555 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-web-config\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553426    2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553435    2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e8d0a641-40fb-4926-8b45-6d71aedcba5f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553443    2555 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e8d0a641-40fb-4926-8b45-6d71aedcba5f-tls-assets\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553451    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e8d0a641-40fb-4926-8b45-6d71aedcba5f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.553481 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.553458    2555 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e8d0a641-40fb-4926-8b45-6d71aedcba5f-config-out\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:09:11.934620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934590    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13" exitCode=0
Jul 15 21:09:11.934620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934614    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171" exitCode=0
Jul 15 21:09:11.934620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934623    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82" exitCode=0
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934634    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d" exitCode=0
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934643    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c" exitCode=0
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934651    2555 generic.go:356] "Generic (PLEG): container finished" podID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995" exitCode=0
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934658    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934669    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"}
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934735    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"}
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934762    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"}
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934786    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"}
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934807    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.934894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934810    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"}
Jul 15 21:09:11.935315 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934913    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"}
Jul 15 21:09:11.935315 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.934940    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e8d0a641-40fb-4926-8b45-6d71aedcba5f","Type":"ContainerDied","Data":"550d2b29727370191c40da903d32398059f7c8088fe478cf5d1719fd6a8dfe71"}
Jul 15 21:09:11.943355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.943334    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.949778 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.949762    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.955656 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.955639    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.961335 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.961289    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:09:11.962174 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.962159    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.967191 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.967174    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:09:11.968846 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.968832    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.975305 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.975290    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.982819 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.982796    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.983278 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.983257    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.983365 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.983285    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"} err="failed to get container status \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist"
Jul 15 21:09:11.983365 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.983316    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.983553 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.983532    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.983603 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.983565    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"} err="failed to get container status \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist"
Jul 15 21:09:11.983603 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.983581    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.983805 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.983791    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.983854 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.983809    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"} err="failed to get container status \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist"
Jul 15 21:09:11.983854 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.983821    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.984085 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.984067    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.984156 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984087    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"} err="failed to get container status \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist"
Jul 15 21:09:11.984156 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984100    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.984361 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.984340    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.984427 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984371    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"} err="failed to get container status \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist"
Jul 15 21:09:11.984427 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984386    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.984611 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.984594    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.984649 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984615    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"} err="failed to get container status \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist"
Jul 15 21:09:11.984649 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984630    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.984841 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:09:11.984827    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.984880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984844    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"} err="failed to get container status \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist"
Jul 15 21:09:11.984880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.984857    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.985120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985099    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"} err="failed to get container status \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist"
Jul 15 21:09:11.985120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985119    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.985349 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985330    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"} err="failed to get container status \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist"
Jul 15 21:09:11.985397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985349    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.985558 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985542    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"} err="failed to get container status \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist"
Jul 15 21:09:11.985601 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985558    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.985776 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985759    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"} err="failed to get container status \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist"
Jul 15 21:09:11.985835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985776    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.985996 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985979    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"} err="failed to get container status \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist"
Jul 15 21:09:11.986044 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.985995    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.986198 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986181    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"} err="failed to get container status \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist"
Jul 15 21:09:11.986240 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986198    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.986406 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986388    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"} err="failed to get container status \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist"
Jul 15 21:09:11.986447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986407    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.986593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986578    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"} err="failed to get container status \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist"
Jul 15 21:09:11.986632 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986593    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.986760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986742    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"} err="failed to get container status \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist"
Jul 15 21:09:11.986811 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986763    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.986999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.986980    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"} err="failed to get container status \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist"
Jul 15 21:09:11.987057 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987000    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.987219 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987202    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"} err="failed to get container status \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist"
Jul 15 21:09:11.987260 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987220    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.987422 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987406    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"} err="failed to get container status \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist"
Jul 15 21:09:11.987422 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987421    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.987616 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987589    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"} err="failed to get container status \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist"
Jul 15 21:09:11.987655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987616    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.987816 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987798    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"} err="failed to get container status \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist"
Jul 15 21:09:11.987856 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.987816    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.988043 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988027    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"} err="failed to get container status \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist"
Jul 15 21:09:11.988099 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988043    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.988247 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988231    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"} err="failed to get container status \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist"
Jul 15 21:09:11.988291 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988247    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.988428 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988412    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"} err="failed to get container status \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist"
Jul 15 21:09:11.988475 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988427    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.988612 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988597    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"} err="failed to get container status \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist"
Jul 15 21:09:11.988662 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988613    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.988787 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988772    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"} err="failed to get container status \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist"
Jul 15 21:09:11.988826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988787    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.988991 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988972    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"} err="failed to get container status \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist"
Jul 15 21:09:11.989048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.988993    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.989168 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989149    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"} err="failed to get container status \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist"
Jul 15 21:09:11.989204 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989169    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.989341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989326    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"} err="failed to get container status \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist"
Jul 15 21:09:11.989382 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989341    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.989541 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989524    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"} err="failed to get container status \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist"
Jul 15 21:09:11.989582 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989541    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.989737 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989718    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"} err="failed to get container status \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist"
Jul 15 21:09:11.989806 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989740    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.989946 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989931    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"} err="failed to get container status \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist"
Jul 15 21:09:11.990012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.989946    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.990123 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990109    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"} err="failed to get container status \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist"
Jul 15 21:09:11.990123 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990122    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.990289 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990272    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"} err="failed to get container status \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist"
Jul 15 21:09:11.990349 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990291    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.990472 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990455    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"} err="failed to get container status \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist"
Jul 15 21:09:11.990523 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990473    2555 scope.go:117] "RemoveContainer" containerID="bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"
Jul 15 21:09:11.990631 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990616    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13"} err="failed to get container status \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": rpc error: code = NotFound desc = could not find container \"bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13\": container with ID starting with bab93d636cf7282db76d94f630d026653b80b2e82b3bb2c62b643bbee6928d13 not found: ID does not exist"
Jul 15 21:09:11.990673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990631    2555 scope.go:117] "RemoveContainer" containerID="6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"
Jul 15 21:09:11.990824 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990810    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171"} err="failed to get container status \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": rpc error: code = NotFound desc = could not find container \"6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171\": container with ID starting with 6f1c123a731677b94d188351e7102265aa73df41eabad2a969c9fff822842171 not found: ID does not exist"
Jul 15 21:09:11.990864 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.990823    2555 scope.go:117] "RemoveContainer" containerID="69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"
Jul 15 21:09:11.991042 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991022    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82"} err="failed to get container status \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": rpc error: code = NotFound desc = could not find container \"69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82\": container with ID starting with 69c4d95de9be8ba1e161f85ec88e6d1f9670fc765f0d7a085a6ba8f118390e82 not found: ID does not exist"
Jul 15 21:09:11.991088 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991043    2555 scope.go:117] "RemoveContainer" containerID="0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"
Jul 15 21:09:11.991224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991204    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d"} err="failed to get container status \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": rpc error: code = NotFound desc = could not find container \"0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d\": container with ID starting with 0cc14bcabc1cd28c0fd47b8c5bbab774dd5ec372adfad77c0a2ba4229eeef04d not found: ID does not exist"
Jul 15 21:09:11.991264 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991224    2555 scope.go:117] "RemoveContainer" containerID="ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"
Jul 15 21:09:11.991405 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991388    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c"} err="failed to get container status \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": rpc error: code = NotFound desc = could not find container \"ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c\": container with ID starting with ec189c152fe3e7c3493752f3a312c563d5396668f66a2cd73fee2648586e6e4c not found: ID does not exist"
Jul 15 21:09:11.991447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991406    2555 scope.go:117] "RemoveContainer" containerID="f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"
Jul 15 21:09:11.991579 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991556    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995"} err="failed to get container status \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": rpc error: code = NotFound desc = could not find container \"f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995\": container with ID starting with f55758c63513461dba9ad87c9e0af7c4a4395f8c1126d1e47651d17a118eb995 not found: ID does not exist"
Jul 15 21:09:11.991634 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991581    2555 scope.go:117] "RemoveContainer" containerID="3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"
Jul 15 21:09:11.991791 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.991773    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e"} err="failed to get container status \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": rpc error: code = NotFound desc = could not find container \"3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e\": container with ID starting with 3a7187befa077bf7af1ea74f8fe8697490b24d1b6acd0a1907cb5e6221b4409e not found: ID does not exist"
Jul 15 21:09:11.995687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995670    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:09:11.995879 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995868    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="config-reloader"
Jul 15 21:09:11.995915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995881    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="config-reloader"
Jul 15 21:09:11.995915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995889    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-thanos"
Jul 15 21:09:11.995915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995896    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-thanos"
Jul 15 21:09:11.995915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995905    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="prometheus"
Jul 15 21:09:11.995915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995910    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="prometheus"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995918    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="thanos-sidecar"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995924    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="thanos-sidecar"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995929    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-web"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995934    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-web"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995945    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="init-config-reloader"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995951    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="init-config-reloader"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995972    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.995976    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.996025    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-thanos"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.996032    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="config-reloader"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.996039    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="thanos-sidecar"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.996043    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.996048    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="prometheus"
Jul 15 21:09:11.996081 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.996052    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" containerName="kube-rbac-proxy-web"
Jul 15 21:09:11.999427 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:11.999412    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.002215 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002194    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 21:09:12.002399 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002382    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8w2mz\""
Jul 15 21:09:12.002478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002424    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 21:09:12.002602 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002589    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 21:09:12.002663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002609    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 21:09:12.002663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002627    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 21:09:12.002968 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002937    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 21:09:12.002968 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002947    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 21:09:12.003095 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.002973    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 21:09:12.003238 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.003216    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 21:09:12.003377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.003362    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 21:09:12.003447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.003369    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1iu1fl5j903tr\""
Jul 15 21:09:12.003447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.003424    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 21:09:12.004176 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.004162    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 21:09:12.004773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.004753    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 21:09:12.013396 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.010763    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 21:09:12.013396 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.012792    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 21:09:12.016627 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.016604    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:09:12.057878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.057856    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058016 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.057881    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058016 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.057913    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058016 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.057929    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058016 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058002    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058023    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058052    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058072    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058088    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058125    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058154    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058175    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058201    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-config\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058217    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058232    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctjnm\" (UniqueName: \"kubernetes.io/projected/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-kube-api-access-ctjnm\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058250    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058277    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058505 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058301    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058505 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058321    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.058505 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.058350    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159163 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159142    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159170    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159187    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-config\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159203    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159228    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctjnm\" (UniqueName: \"kubernetes.io/projected/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-kube-api-access-ctjnm\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159253    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159275    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159300    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159328    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159359    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159383    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159405    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159446    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159471    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.159507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159504    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159525    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159550    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159574    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159603    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.159634    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.160004    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.160110    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160226 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.160110    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.160477    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.160623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.160525    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.161231 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.161204    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.162441 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.162329    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.162537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.162439    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.162537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.162439    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.162537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.162446    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.162983 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.162745    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.163223 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.163152    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.163749 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.163694    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-config\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.163925 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.163883    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.164711 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.164686    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.165111 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.165088    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.165311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.165292    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.165394 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.165373    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.165715 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.165698    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.168209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.168191    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctjnm\" (UniqueName: \"kubernetes.io/projected/3c8ec6a8-4307-4f88-a405-4f0a10345e2b-kube-api-access-ctjnm\") pod \"prometheus-k8s-0\" (UID: \"3c8ec6a8-4307-4f88-a405-4f0a10345e2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.215657 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.215606    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8d0a641-40fb-4926-8b45-6d71aedcba5f" path="/var/lib/kubelet/pods/e8d0a641-40fb-4926-8b45-6d71aedcba5f/volumes"
Jul 15 21:09:12.309382 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.309360    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:12.437478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.437393    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 21:09:12.439860 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:09:12.439832    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c8ec6a8_4307_4f88_a405_4f0a10345e2b.slice/crio-99981ba3dc4e9bb2f4ee8eb0b0296122c9432ca6053544fc20854757c8564629 WatchSource:0}: Error finding container 99981ba3dc4e9bb2f4ee8eb0b0296122c9432ca6053544fc20854757c8564629: Status 404 returned error can't find the container with id 99981ba3dc4e9bb2f4ee8eb0b0296122c9432ca6053544fc20854757c8564629
Jul 15 21:09:12.939758 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.939731    2555 generic.go:356] "Generic (PLEG): container finished" podID="3c8ec6a8-4307-4f88-a405-4f0a10345e2b" containerID="93f45da42dcb7c1dc97e37a6587c02170fdbb022e42420a2714ff3f6dd748ead" exitCode=0
Jul 15 21:09:12.939905 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.939822    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerDied","Data":"93f45da42dcb7c1dc97e37a6587c02170fdbb022e42420a2714ff3f6dd748ead"}
Jul 15 21:09:12.939992 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:12.939910    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"99981ba3dc4e9bb2f4ee8eb0b0296122c9432ca6053544fc20854757c8564629"}
Jul 15 21:09:13.947891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.947857    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"64f7ba9bb3b4e276fb4bfaa2be7d0cc21f9369cfc5c71a78cc2a4806af8f4cef"}
Jul 15 21:09:13.948383 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.947919    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"8901529b4140ab6dc4d956b0988ae85790bb04c3212239f40e0a9871e4508474"}
Jul 15 21:09:13.948383 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.947945    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"476f184352b61402c911f9bab436c774006eb0d92bcf147710aaac1aa322f35c"}
Jul 15 21:09:13.948383 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.947991    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"ca6dd1a07d11d46e6a1fb86694f48289a92ce039fd66cb77ce6fc6b01aa1ae17"}
Jul 15 21:09:13.948383 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.948016    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"ac52baf73574b7c37d435dda1885c2f3a476c8a46be97e45683b8cd780f104fd"}
Jul 15 21:09:13.948383 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.948038    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3c8ec6a8-4307-4f88-a405-4f0a10345e2b","Type":"ContainerStarted","Data":"4362364634b7cb271356d03c990134032f5d3392f4f732095e4bb954c0008ee6"}
Jul 15 21:09:13.980583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:13.980538    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.961737016 podStartE2EDuration="2.980523188s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.786172ms" isStatefulPod=false podCreationTimestamp="2026-07-15 21:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:09:13.978656157 +0000 UTC m=+158.310797259" watchObservedRunningTime="2026-07-15 21:09:13.980523188 +0000 UTC m=+158.312664277"
Jul 15 21:09:14.140887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:14.140852    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:15.128209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:15.128157    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:15.325208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:15.325105    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:15.426483 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:15.426441    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:15.864684 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:15.864654    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:15.922699 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:15.922672    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:17.309603 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:17.309560    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:09:26.320181 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:26.320146    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:26.379309 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:26.379279    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:26.439101 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:26.439073    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:09:57.753366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:09:57.753333    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:12.310077 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:12.310029    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:10:12.325136 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:12.325112    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:10:13.162825 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:13.162800    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 21:10:21.007312 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:21.007273    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:21.080208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:21.080178    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:32.925449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:32.925417    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:33.028232 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:33.028183    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:35.525795 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:35.525761    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:46.662040 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:46.662009    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:46.719156 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:46.719125    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:49.574641 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:49.574607    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:49.668209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:49.668173    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:10:49.730976 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:10:49.730923    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:00.212189 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:00.212093    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:00.271162 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:00.271127    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:20.403891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:20.403846    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:20.513264 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:20.513227    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:36.055195 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:36.055168    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:11:36.058370 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:36.058347    2555 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 21:11:36.060129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:36.060112    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:11:42.047659 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:42.047613    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:43.393003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:43.392975    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:45.628857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:45.628827    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:45.728815 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:45.728773    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:48.136352 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:48.136318    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:48.244172 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:48.244130    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:48.350581 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:48.350544    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:48.464878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:48.464783    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:50.700383 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:50.700345    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:50.858762 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:50.858726    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:52.447856 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:52.447831    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:52.509783 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:52.509740    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:11:58.478730 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:11:58.478701    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:00.729519 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:00.729478    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:10.431790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:10.431758    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:10.530742 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:10.530690    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:14.036607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:14.036578    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:14.137124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:14.137094    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:16.371664 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:16.371626    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:16.472995 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:16.472950    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:18.606549 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:18.606515    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:22.903966 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:22.903933    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:22.975353 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:22.975318    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:34.438877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:34.438805    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:34.510214 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:34.510182    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:34.982411 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:34.982381    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.331078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.331003    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.404533 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.404479    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.440015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.439983    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"]
Jul 15 21:12:35.443102 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.443065    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.446347 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.446323    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 21:12:35.446461 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.446364    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zg2b8\""
Jul 15 21:12:35.446587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.446572    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 21:12:35.446826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.446811    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 21:12:35.446910 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.446839    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 21:12:35.459680 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.459656    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"]
Jul 15 21:12:35.606668 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.606602    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42978d06-f6bc-4e7b-9ca4-83606ade276f-tmp\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.606668 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.606633    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42978d06-f6bc-4e7b-9ca4-83606ade276f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.606668 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.606656    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42978d06-f6bc-4e7b-9ca4-83606ade276f-webhook-cert\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.606846 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.606713    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42978d06-f6bc-4e7b-9ca4-83606ade276f-manifests\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.606846 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.606739    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn695\" (UniqueName: \"kubernetes.io/projected/42978d06-f6bc-4e7b-9ca4-83606ade276f-kube-api-access-xn695\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.612720 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.612696    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.707448 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.707421    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42978d06-f6bc-4e7b-9ca4-83606ade276f-manifests\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.707567 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.707455    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn695\" (UniqueName: \"kubernetes.io/projected/42978d06-f6bc-4e7b-9ca4-83606ade276f-kube-api-access-xn695\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.707567 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.707501    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42978d06-f6bc-4e7b-9ca4-83606ade276f-tmp\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.707567 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.707519    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42978d06-f6bc-4e7b-9ca4-83606ade276f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.707567 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.707536    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42978d06-f6bc-4e7b-9ca4-83606ade276f-webhook-cert\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.707854 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.707831    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42978d06-f6bc-4e7b-9ca4-83606ade276f-manifests\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.709831 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.709801    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42978d06-f6bc-4e7b-9ca4-83606ade276f-tmp\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.710017 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.710001    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42978d06-f6bc-4e7b-9ca4-83606ade276f-webhook-cert\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.710075 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.710003    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42978d06-f6bc-4e7b-9ca4-83606ade276f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.711139 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.711114    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.718021 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.718001    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn695\" (UniqueName: \"kubernetes.io/projected/42978d06-f6bc-4e7b-9ca4-83606ade276f-kube-api-access-xn695\") pod \"opendatahub-operator-controller-manager-858d4f578f-rtk74\" (UID: \"42978d06-f6bc-4e7b-9ca4-83606ade276f\") " pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.753859 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.753837    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:35.814268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.814220    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.895451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.895423    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:35.895864 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.895738    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"]
Jul 15 21:12:35.898787 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:12:35.898761    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42978d06_f6bc_4e7b_9ca4_83606ade276f.slice/crio-8b78cafc0065bf108da15b1bc30bb814c85abee42f6b000cb0004f73c6566f29 WatchSource:0}: Error finding container 8b78cafc0065bf108da15b1bc30bb814c85abee42f6b000cb0004f73c6566f29: Status 404 returned error can't find the container with id 8b78cafc0065bf108da15b1bc30bb814c85abee42f6b000cb0004f73c6566f29
Jul 15 21:12:35.900813 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.900798    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:12:35.975808 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:35.975788    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:36.077928 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:36.077904    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:36.179486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:36.179416    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:36.611689 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:36.611597    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" event={"ID":"42978d06-f6bc-4e7b-9ca4-83606ade276f","Type":"ContainerStarted","Data":"8b78cafc0065bf108da15b1bc30bb814c85abee42f6b000cb0004f73c6566f29"}
Jul 15 21:12:37.886171 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:37.886129    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:37.984848 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:37.984819    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:39.626385 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:39.626343    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" event={"ID":"42978d06-f6bc-4e7b-9ca4-83606ade276f","Type":"ContainerStarted","Data":"165c167797ac05ef31ce352d6ed1d031db3418bba48a85dd8ae61b8551c02c2f"}
Jul 15 21:12:40.631627 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:40.631596    2555 generic.go:356] "Generic (PLEG): container finished" podID="42978d06-f6bc-4e7b-9ca4-83606ade276f" containerID="165c167797ac05ef31ce352d6ed1d031db3418bba48a85dd8ae61b8551c02c2f" exitCode=0
Jul 15 21:12:40.632049 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:40.631690    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" event={"ID":"42978d06-f6bc-4e7b-9ca4-83606ade276f","Type":"ContainerDied","Data":"165c167797ac05ef31ce352d6ed1d031db3418bba48a85dd8ae61b8551c02c2f"}
Jul 15 21:12:41.637687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:41.637650    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" event={"ID":"42978d06-f6bc-4e7b-9ca4-83606ade276f","Type":"ContainerStarted","Data":"9c7a482fedc45a27780d2ae8d257e050eccbd8217b982760e78d1bc31d1bdd48"}
Jul 15 21:12:41.638184 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:41.637806    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:41.667129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:41.667083    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" podStartSLOduration=2.3270637929999998 podStartE2EDuration="6.667068952s" totalImagesPullingTime="3.262020358s" totalInitContainerRuntime="1.077984801s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:12:41.665302045 +0000 UTC m=+365.997443165" watchObservedRunningTime="2026-07-15 21:12:41.667068952 +0000 UTC m=+365.999210041"
Jul 15 21:12:46.250106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:46.250073    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:46.302318 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:46.302286    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:52.643524 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:52.643494    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:12:54.388317 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:54.388277    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:54.745859 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:54.745773    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:54.923923 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:54.923888    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:55.137589 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.137549    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"]
Jul 15 21:12:55.147738 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.147712    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.151008 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.150977    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 21:12:55.151370 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.151347    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 21:12:55.151581 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.151564    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 21:12:55.151735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.151699    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 21:12:55.151735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.151347    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fj9vd\""
Jul 15 21:12:55.152078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.152056    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"]
Jul 15 21:12:55.252233 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.252201    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d05f479c-075c-4e14-902f-6c3571e0418d-tmp\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.252233 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.252236    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sgq7\" (UniqueName: \"kubernetes.io/projected/d05f479c-075c-4e14-902f-6c3571e0418d-kube-api-access-5sgq7\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.252471 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.252263    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05f479c-075c-4e14-902f-6c3571e0418d-tls-certs\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.352764 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.352731    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d05f479c-075c-4e14-902f-6c3571e0418d-tmp\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.352913 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.352770    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sgq7\" (UniqueName: \"kubernetes.io/projected/d05f479c-075c-4e14-902f-6c3571e0418d-kube-api-access-5sgq7\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.352913 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.352813    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05f479c-075c-4e14-902f-6c3571e0418d-tls-certs\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.353003 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:12:55.352947    2555 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 15 21:12:55.353065 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:12:55.353054    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05f479c-075c-4e14-902f-6c3571e0418d-tls-certs podName:d05f479c-075c-4e14-902f-6c3571e0418d nodeName:}" failed. No retries permitted until 2026-07-15 21:12:55.853033714 +0000 UTC m=+380.185174797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d05f479c-075c-4e14-902f-6c3571e0418d-tls-certs") pod "kube-auth-proxy-5bcb5978cd-w5cpd" (UID: "d05f479c-075c-4e14-902f-6c3571e0418d") : secret "kube-auth-proxy-tls" not found
Jul 15 21:12:55.355220 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.355193    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d05f479c-075c-4e14-902f-6c3571e0418d-tmp\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.363622 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.363597    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j5z24"]
Jul 15 21:12:55.365735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.365719    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:55.372704 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.372682    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sgq7\" (UniqueName: \"kubernetes.io/projected/d05f479c-075c-4e14-902f-6c3571e0418d-kube-api-access-5sgq7\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.374001 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.373981    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 21:12:55.374103 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.374087    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-75gch\""
Jul 15 21:12:55.408867 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.408798    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j5z24"]
Jul 15 21:12:55.427878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.427850    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:55.453429 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.453396    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw9t9\" (UniqueName: \"kubernetes.io/projected/e87f8b62-a84a-479d-b8ea-44ec05e51993-kube-api-access-rw9t9\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:55.453556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.453436    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e87f8b62-a84a-479d-b8ea-44ec05e51993-cert\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:55.519366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.519339    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:55.554606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.554569    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw9t9\" (UniqueName: \"kubernetes.io/projected/e87f8b62-a84a-479d-b8ea-44ec05e51993-kube-api-access-rw9t9\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:55.554794 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.554616    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e87f8b62-a84a-479d-b8ea-44ec05e51993-cert\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:55.554794 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:12:55.554730    2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 21:12:55.554923 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:12:55.554810    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87f8b62-a84a-479d-b8ea-44ec05e51993-cert podName:e87f8b62-a84a-479d-b8ea-44ec05e51993 nodeName:}" failed. No retries permitted until 2026-07-15 21:12:56.054786891 +0000 UTC m=+380.386927963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e87f8b62-a84a-479d-b8ea-44ec05e51993-cert") pod "odh-model-controller-7998c67dc4-j5z24" (UID: "e87f8b62-a84a-479d-b8ea-44ec05e51993") : secret "odh-model-controller-webhook-cert" not found
Jul 15 21:12:55.586707 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.586675    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw9t9\" (UniqueName: \"kubernetes.io/projected/e87f8b62-a84a-479d-b8ea-44ec05e51993-kube-api-access-rw9t9\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:55.625423 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.625397    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:55.782296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.782209    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:55.857173 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.857137    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05f479c-075c-4e14-902f-6c3571e0418d-tls-certs\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:55.859613 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:55.859589    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05f479c-075c-4e14-902f-6c3571e0418d-tls-certs\") pod \"kube-auth-proxy-5bcb5978cd-w5cpd\" (UID: \"d05f479c-075c-4e14-902f-6c3571e0418d\") " pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:56.058817 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.058731    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e87f8b62-a84a-479d-b8ea-44ec05e51993-cert\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:56.060883 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.060851    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"
Jul 15 21:12:56.061224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.061203    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e87f8b62-a84a-479d-b8ea-44ec05e51993-cert\") pod \"odh-model-controller-7998c67dc4-j5z24\" (UID: \"e87f8b62-a84a-479d-b8ea-44ec05e51993\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:56.196802 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:12:56.196772    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05f479c_075c_4e14_902f_6c3571e0418d.slice/crio-3111bd4e3813042cf8b0b1be1eeccba03a5dbd7c1e2c6a6c2bffef6d63df855d WatchSource:0}: Error finding container 3111bd4e3813042cf8b0b1be1eeccba03a5dbd7c1e2c6a6c2bffef6d63df855d: Status 404 returned error can't find the container with id 3111bd4e3813042cf8b0b1be1eeccba03a5dbd7c1e2c6a6c2bffef6d63df855d
Jul 15 21:12:56.206561 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.206539    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd"]
Jul 15 21:12:56.281446 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.281412    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:12:56.345891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.345859    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:56.366163 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.366133    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-prn5d"]
Jul 15 21:12:56.368948 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.368926    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:56.371820 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.371802    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 21:12:56.380880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.380820    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-prn5d"]
Jul 15 21:12:56.435404 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.435376    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j5z24"]
Jul 15 21:12:56.436510 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:12:56.436480    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87f8b62_a84a_479d_b8ea_44ec05e51993.slice/crio-3fdfb12ce29b21a84415e6b6478dba1f7d5e4b0f3e069a55391b1eb462d224a2 WatchSource:0}: Error finding container 3fdfb12ce29b21a84415e6b6478dba1f7d5e4b0f3e069a55391b1eb462d224a2: Status 404 returned error can't find the container with id 3fdfb12ce29b21a84415e6b6478dba1f7d5e4b0f3e069a55391b1eb462d224a2
Jul 15 21:12:56.462449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.462424    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9564402b-7e1e-4275-aee5-adbe06c3c21b-tls-certs\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:56.462583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.462466    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7lwf\" (UniqueName: \"kubernetes.io/projected/9564402b-7e1e-4275-aee5-adbe06c3c21b-kube-api-access-w7lwf\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:56.563097 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.563063    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9564402b-7e1e-4275-aee5-adbe06c3c21b-tls-certs\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:56.563274 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.563113    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7lwf\" (UniqueName: \"kubernetes.io/projected/9564402b-7e1e-4275-aee5-adbe06c3c21b-kube-api-access-w7lwf\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:56.563274 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:12:56.563214    2555 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 15 21:12:56.563274 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:12:56.563274    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9564402b-7e1e-4275-aee5-adbe06c3c21b-tls-certs podName:9564402b-7e1e-4275-aee5-adbe06c3c21b nodeName:}" failed. No retries permitted until 2026-07-15 21:12:57.063258101 +0000 UTC m=+381.395399170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9564402b-7e1e-4275-aee5-adbe06c3c21b-tls-certs") pod "model-serving-api-54d89bd5fc-prn5d" (UID: "9564402b-7e1e-4275-aee5-adbe06c3c21b") : secret "model-serving-api-tls" not found
Jul 15 21:12:56.573873 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.573843    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7lwf\" (UniqueName: \"kubernetes.io/projected/9564402b-7e1e-4275-aee5-adbe06c3c21b-kube-api-access-w7lwf\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:56.615085 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.615008    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:56.629604 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.629575    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"]
Jul 15 21:12:56.631822 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.631799    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.634950 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.634922    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 21:12:56.634950 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.634945    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 21:12:56.635128 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.634922    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 21:12:56.635267 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.635251    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 21:12:56.635684 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.635670    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-72hmm\""
Jul 15 21:12:56.663573 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663532    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-cacerts\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.663732 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663611    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-local-certs\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.663732 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663647    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.663732 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663705    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.663898 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663751    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsxhf\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-kube-api-access-wsxhf\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.663898 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663833    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-token\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.664020 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.663901    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.689484 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.689456    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd" event={"ID":"d05f479c-075c-4e14-902f-6c3571e0418d","Type":"ContainerStarted","Data":"3111bd4e3813042cf8b0b1be1eeccba03a5dbd7c1e2c6a6c2bffef6d63df855d"}
Jul 15 21:12:56.690494 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.690474    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24" event={"ID":"e87f8b62-a84a-479d-b8ea-44ec05e51993","Type":"ContainerStarted","Data":"3fdfb12ce29b21a84415e6b6478dba1f7d5e4b0f3e069a55391b1eb462d224a2"}
Jul 15 21:12:56.728746 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.728715    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"]
Jul 15 21:12:56.734705 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.734683    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:56.765051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765016    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765061    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765092    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsxhf\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-kube-api-access-wsxhf\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765138    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-token\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765180    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765469 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765231    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-cacerts\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765469 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765277    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-local-certs\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.765741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.765711    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.767722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.767662    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-local-certs\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.767845 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.767824    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-cacerts\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.768025 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.768002    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.768332 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.768315    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.778103 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.778070    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-token\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.778346 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.778326    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsxhf\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-kube-api-access-wsxhf\") pod \"istiod-openshift-gateway-7574459656-vcv9r\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:56.834548 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.834512    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:12:56.941758 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:56.941727    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:12:57.066986 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.066929    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9564402b-7e1e-4275-aee5-adbe06c3c21b-tls-certs\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:57.070339 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.070314    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9564402b-7e1e-4275-aee5-adbe06c3c21b-tls-certs\") pod \"model-serving-api-54d89bd5fc-prn5d\" (UID: \"9564402b-7e1e-4275-aee5-adbe06c3c21b\") " pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:57.103662 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.103466    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"]
Jul 15 21:12:57.106442 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:12:57.106414    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2a2eed_5d2d_4b96_8e54_d1614fd3233c.slice/crio-1c2c417a741384137833b57cae99b08486a202aae3f1a837673d76582e7e5f28 WatchSource:0}: Error finding container 1c2c417a741384137833b57cae99b08486a202aae3f1a837673d76582e7e5f28: Status 404 returned error can't find the container with id 1c2c417a741384137833b57cae99b08486a202aae3f1a837673d76582e7e5f28
Jul 15 21:12:57.281947 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.281773    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:12:57.463874 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.463754    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-prn5d"]
Jul 15 21:12:57.468675 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:12:57.468621    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9564402b_7e1e_4275_aee5_adbe06c3c21b.slice/crio-d094a371bd2ecec027f150e7cf41422bdb30800a0eee3b5cc71e5430dd715fdd WatchSource:0}: Error finding container d094a371bd2ecec027f150e7cf41422bdb30800a0eee3b5cc71e5430dd715fdd: Status 404 returned error can't find the container with id d094a371bd2ecec027f150e7cf41422bdb30800a0eee3b5cc71e5430dd715fdd
Jul 15 21:12:57.698303 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.698264    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" event={"ID":"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c","Type":"ContainerStarted","Data":"1c2c417a741384137833b57cae99b08486a202aae3f1a837673d76582e7e5f28"}
Jul 15 21:12:57.699652 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.699627    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d" event={"ID":"9564402b-7e1e-4275-aee5-adbe06c3c21b","Type":"ContainerStarted","Data":"d094a371bd2ecec027f150e7cf41422bdb30800a0eee3b5cc71e5430dd715fdd"}
Jul 15 21:12:57.982356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:12:57.982241    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:01.741990 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:01.741935    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:13:01.742401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:01.742034    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:13:02.465048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.465014    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:02.589741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.589701    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:02.724238 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.724095    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd" event={"ID":"d05f479c-075c-4e14-902f-6c3571e0418d","Type":"ContainerStarted","Data":"2d9e6f696cf0629bf74af35f4c5943ff861b722da1b0189fca991fc641e2a771"}
Jul 15 21:13:02.726019 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.725989    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" event={"ID":"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c","Type":"ContainerStarted","Data":"a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9"}
Jul 15 21:13:02.726181 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.726052    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:13:02.728035 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.728000    2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vcv9r container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 21:13:02.728134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.728053    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" podUID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 21:13:02.728176 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.728150    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d" event={"ID":"9564402b-7e1e-4275-aee5-adbe06c3c21b","Type":"ContainerStarted","Data":"ba517d9d28efec807284d9664ffab9e6879bcefa1bded401ba92395a34e91956"}
Jul 15 21:13:02.728270 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.728253    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:13:02.729588 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.729565    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24" event={"ID":"e87f8b62-a84a-479d-b8ea-44ec05e51993","Type":"ContainerStarted","Data":"2294aacf41b289ecab5e2d9c0b001670a65b318b44cc69ef77bd9da754b0cc0f"}
Jul 15 21:13:02.729709 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.729695    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:13:02.746175 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.746120    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bcb5978cd-w5cpd" podStartSLOduration=2.22388644 podStartE2EDuration="7.746101522s" totalImagesPullingTime="5.522215082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:02.745248458 +0000 UTC m=+387.077389568" watchObservedRunningTime="2026-07-15 21:13:02.746101522 +0000 UTC m=+387.078242615"
Jul 15 21:13:02.771913 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.771857    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24" podStartSLOduration=2.459569203 podStartE2EDuration="7.771839156s" totalImagesPullingTime="5.312269953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:02.769859259 +0000 UTC m=+387.102000352" watchObservedRunningTime="2026-07-15 21:13:02.771839156 +0000 UTC m=+387.103980245"
Jul 15 21:13:02.793112 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.793061    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d" podStartSLOduration=2.5414598330000002 podStartE2EDuration="6.793045962s" totalImagesPullingTime="4.251586129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:02.792000516 +0000 UTC m=+387.124141635" watchObservedRunningTime="2026-07-15 21:13:02.793045962 +0000 UTC m=+387.125187052"
Jul 15 21:13:02.819179 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.819132    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" podStartSLOduration=2.186396254 podStartE2EDuration="6.819116666s" totalImagesPullingTime="4.632720412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:02.81758308 +0000 UTC m=+387.149724274" watchObservedRunningTime="2026-07-15 21:13:02.819116666 +0000 UTC m=+387.151257755"
Jul 15 21:13:02.888061 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:02.888026    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:03.120091 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:03.120006    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:03.413487 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:03.413429    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:03.734623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:03.734541    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:13:07.451151 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.451116    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:07.548113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.548080    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:07.559166 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.559128    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"]
Jul 15 21:13:07.561408 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.561393    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"]
Jul 15 21:13:07.561546 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.561524    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:07.563324 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.563306    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:07.570265 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.569932    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2ws4n\""
Jul 15 21:13:07.570265 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.570100    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 15 21:13:07.570265 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.570169    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 21:13:07.570609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.570591    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sljjg\""
Jul 15 21:13:07.575039 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.575016    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"]
Jul 15 21:13:07.586254 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.586220    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"]
Jul 15 21:13:07.622851 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.622797    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:07.660115 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.660082    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5vbh\" (UniqueName: \"kubernetes.io/projected/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-kube-api-access-k5vbh\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:07.660115 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.660130    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-cert\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:07.660366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.660159    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550cae76-7a34-4e8a-bad1-caca322ec031-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:07.660366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.660209    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqw86\" (UniqueName: \"kubernetes.io/projected/550cae76-7a34-4e8a-bad1-caca322ec031-kube-api-access-tqw86\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:07.760922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.760838    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5vbh\" (UniqueName: \"kubernetes.io/projected/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-kube-api-access-k5vbh\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:07.760922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.760881    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-cert\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:07.760922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.760915    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550cae76-7a34-4e8a-bad1-caca322ec031-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:07.761215 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.760936    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqw86\" (UniqueName: \"kubernetes.io/projected/550cae76-7a34-4e8a-bad1-caca322ec031-kube-api-access-tqw86\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:07.761215 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:13:07.761051    2555 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 15 21:13:07.761215 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:13:07.761069    2555 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 15 21:13:07.761215 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:13:07.761123    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-cert podName:b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89 nodeName:}" failed. No retries permitted until 2026-07-15 21:13:08.261103008 +0000 UTC m=+392.593244089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-cert") pod "kserve-controller-manager-d6f4b65b8-6pscc" (UID: "b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89") : secret "kserve-webhook-server-cert" not found
Jul 15 21:13:07.761215 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:13:07.761143    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550cae76-7a34-4e8a-bad1-caca322ec031-cert podName:550cae76-7a34-4e8a-bad1-caca322ec031 nodeName:}" failed. No retries permitted until 2026-07-15 21:13:08.261133366 +0000 UTC m=+392.593274436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/550cae76-7a34-4e8a-bad1-caca322ec031-cert") pod "llmisvc-controller-manager-777c6bd7d5-5s7kk" (UID: "550cae76-7a34-4e8a-bad1-caca322ec031") : secret "llmisvc-webhook-server-cert" not found
Jul 15 21:13:07.771609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.771585    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqw86\" (UniqueName: \"kubernetes.io/projected/550cae76-7a34-4e8a-bad1-caca322ec031-kube-api-access-tqw86\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:07.771730 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.771668    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5vbh\" (UniqueName: \"kubernetes.io/projected/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-kube-api-access-k5vbh\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:07.845306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:07.845250    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:08.121723 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.121626    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:08.266059 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.266006    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-cert\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:08.266228 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.266076    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550cae76-7a34-4e8a-bad1-caca322ec031-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:08.268449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.268427    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550cae76-7a34-4e8a-bad1-caca322ec031-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5s7kk\" (UID: \"550cae76-7a34-4e8a-bad1-caca322ec031\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:08.268560 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.268450    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89-cert\") pod \"kserve-controller-manager-d6f4b65b8-6pscc\" (UID: \"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:08.477221 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.477174    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:08.482499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.482471    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:08.619089 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.619061    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"]
Jul 15 21:13:08.620751 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:13:08.620727    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod550cae76_7a34_4e8a_bad1_caca322ec031.slice/crio-eb05ab630310b90c68b88d608f1090e23458b8c13661dae7e5e37d837b9b5e10 WatchSource:0}: Error finding container eb05ab630310b90c68b88d608f1090e23458b8c13661dae7e5e37d837b9b5e10: Status 404 returned error can't find the container with id eb05ab630310b90c68b88d608f1090e23458b8c13661dae7e5e37d837b9b5e10
Jul 15 21:13:08.633423 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.633399    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"]
Jul 15 21:13:08.636111 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:13:08.636088    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a8a0c6_a572_4aa5_bb46_cb0821bbbc89.slice/crio-ef4c4b05e724cc0a15ff596844a0271e9b2815a66653f871dae1bc71ce57807a WatchSource:0}: Error finding container ef4c4b05e724cc0a15ff596844a0271e9b2815a66653f871dae1bc71ce57807a: Status 404 returned error can't find the container with id ef4c4b05e724cc0a15ff596844a0271e9b2815a66653f871dae1bc71ce57807a
Jul 15 21:13:08.754725 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.754650    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc" event={"ID":"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89","Type":"ContainerStarted","Data":"ef4c4b05e724cc0a15ff596844a0271e9b2815a66653f871dae1bc71ce57807a"}
Jul 15 21:13:08.755794 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.755764    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk" event={"ID":"550cae76-7a34-4e8a-bad1-caca322ec031","Type":"ContainerStarted","Data":"eb05ab630310b90c68b88d608f1090e23458b8c13661dae7e5e37d837b9b5e10"}
Jul 15 21:13:08.761896 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.761876    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:08.864509 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.864478    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:08.972249 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:08.972218    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:10.160407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.159613    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"]
Jul 15 21:13:10.160407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.159851    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" podUID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" containerName="discovery" containerID="cri-o://a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9" gracePeriod=30
Jul 15 21:13:10.430731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.430701    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:13:10.495570 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495541    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-local-certs\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.495757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495578    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-token\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.495757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495616    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-kubeconfig\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.495757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495658    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-cacerts\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.495757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495699    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsxhf\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-kube-api-access-wsxhf\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.495757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495719    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-dns-cert\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.495757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.495753    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-ca-configmap\") pod \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\" (UID: \"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c\") "
Jul 15 21:13:10.496554 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.496525    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:13:10.498676 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.498638    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:13:10.498802 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.498774    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-cacerts" (OuterVolumeSpecName: "cacerts") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:13:10.498890 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.498870    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-local-certs" (OuterVolumeSpecName: "local-certs") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:13:10.498947 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.498902    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-kube-api-access-wsxhf" (OuterVolumeSpecName: "kube-api-access-wsxhf") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "kube-api-access-wsxhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:13:10.498947 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.498908    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:13:10.499394 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.499369    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" (UID: "fb2a2eed-5d2d-4b96-8e54-d1614fd3233c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:13:10.596968 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.596919    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsxhf\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-kube-api-access-wsxhf\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.597138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.596980    2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-dns-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.597138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.596994    2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-csr-ca-configmap\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.597138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.597008    2555 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-local-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.597138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.597022    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.597138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.597034    2555 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-istio-kubeconfig\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.597138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.597045    2555 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c-cacerts\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:10.767268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.767098    2555 generic.go:356] "Generic (PLEG): container finished" podID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" containerID="a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9" exitCode=0
Jul 15 21:13:10.767268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.767155    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"
Jul 15 21:13:10.767268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.767166    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" event={"ID":"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c","Type":"ContainerDied","Data":"a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9"}
Jul 15 21:13:10.767268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.767221    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r" event={"ID":"fb2a2eed-5d2d-4b96-8e54-d1614fd3233c","Type":"ContainerDied","Data":"1c2c417a741384137833b57cae99b08486a202aae3f1a837673d76582e7e5f28"}
Jul 15 21:13:10.767268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.767248    2555 scope.go:117] "RemoveContainer" containerID="a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9"
Jul 15 21:13:10.802123 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.802093    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"]
Jul 15 21:13:10.811970 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.811931    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vcv9r"]
Jul 15 21:13:10.978553 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.978528    2555 scope.go:117] "RemoveContainer" containerID="a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9"
Jul 15 21:13:10.978919 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:13:10.978895    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9\": container with ID starting with a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9 not found: ID does not exist" containerID="a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9"
Jul 15 21:13:10.979079 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:10.978935    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9"} err="failed to get container status \"a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9\": rpc error: code = NotFound desc = could not find container \"a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9\": container with ID starting with a3544fd6a47d6b82b226df5230207fe83e338b79fb2f3f5fe37e74a4c24024c9 not found: ID does not exist"
Jul 15 21:13:11.163332 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.163298    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:11.262643 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.262602    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:11.772276 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.772236    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc" event={"ID":"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89","Type":"ContainerStarted","Data":"5e9a8e3fa9f150546034075574096eb8c4891beca27ed9c81bf77f6273186ea6"}
Jul 15 21:13:11.772461 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.772300    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:11.774536 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.774510    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk" event={"ID":"550cae76-7a34-4e8a-bad1-caca322ec031","Type":"ContainerStarted","Data":"8176ee92397379309c8e61a90634ae55f66b2fdfb54f868117d4029db53d0977"}
Jul 15 21:13:11.774637 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.774620    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:11.792923 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.792876    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc" podStartSLOduration=1.852184048 podStartE2EDuration="4.792862508s" totalImagesPullingTime="2.94067846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:11.792118152 +0000 UTC m=+396.124259266" watchObservedRunningTime="2026-07-15 21:13:11.792862508 +0000 UTC m=+396.125003598"
Jul 15 21:13:11.810628 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:11.810581    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk" podStartSLOduration=1.851003635 podStartE2EDuration="4.810566565s" totalImagesPullingTime="2.95956293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:11.809606053 +0000 UTC m=+396.141747188" watchObservedRunningTime="2026-07-15 21:13:11.810566565 +0000 UTC m=+396.142707688"
Jul 15 21:13:12.216786 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:12.216755    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" path="/var/lib/kubelet/pods/fb2a2eed-5d2d-4b96-8e54-d1614fd3233c/volumes"
Jul 15 21:13:13.736308 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:13.736277    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:13:13.738722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:13.738703    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-prn5d"
Jul 15 21:13:42.782670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:42.782635    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:13:42.785726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:42.785708    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:13:55.982121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:55.982084    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:56.094570 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.094512    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:56.143607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.143577    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpjjb"]
Jul 15 21:13:56.143887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.143870    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" containerName="discovery"
Jul 15 21:13:56.143887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.143888    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" containerName="discovery"
Jul 15 21:13:56.144045 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.143995    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb2a2eed-5d2d-4b96-8e54-d1614fd3233c" containerName="discovery"
Jul 15 21:13:56.158492 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.158466    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:56.159856 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.159831    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpjjb"]
Jul 15 21:13:56.161559 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.161535    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 21:13:56.162283 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.162084    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cpb82\""
Jul 15 21:13:56.163447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.162667    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 21:13:56.194190 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.194163    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:13:56.261760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.261701    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-245v6\" (UniqueName: \"kubernetes.io/projected/2c308d9e-f92e-4808-ac06-b352826f453b-kube-api-access-245v6\") pod \"kuadrant-operator-catalog-bpjjb\" (UID: \"2c308d9e-f92e-4808-ac06-b352826f453b\") " pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:56.362810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.362777    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-245v6\" (UniqueName: \"kubernetes.io/projected/2c308d9e-f92e-4808-ac06-b352826f453b-kube-api-access-245v6\") pod \"kuadrant-operator-catalog-bpjjb\" (UID: \"2c308d9e-f92e-4808-ac06-b352826f453b\") " pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:56.372209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.372176    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-245v6\" (UniqueName: \"kubernetes.io/projected/2c308d9e-f92e-4808-ac06-b352826f453b-kube-api-access-245v6\") pod \"kuadrant-operator-catalog-bpjjb\" (UID: \"2c308d9e-f92e-4808-ac06-b352826f453b\") " pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:56.473131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.473105    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpjjb"]
Jul 15 21:13:56.473329 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.473317    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:56.597761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.597736    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpjjb"]
Jul 15 21:13:56.599770 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:13:56.599741    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c308d9e_f92e_4808_ac06_b352826f453b.slice/crio-503484846d70032ee61f7a9e0e65b8c2220bb4e2e96861a29b6a4b7bb9059edc WatchSource:0}: Error finding container 503484846d70032ee61f7a9e0e65b8c2220bb4e2e96861a29b6a4b7bb9059edc: Status 404 returned error can't find the container with id 503484846d70032ee61f7a9e0e65b8c2220bb4e2e96861a29b6a4b7bb9059edc
Jul 15 21:13:56.684491 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.684466    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtp2s"]
Jul 15 21:13:56.689230 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.689213    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:13:56.697858 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.697831    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtp2s"]
Jul 15 21:13:56.766923 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.766887    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcqv4\" (UniqueName: \"kubernetes.io/projected/80fcc130-f349-4cf8-81bc-2d11a079fae0-kube-api-access-xcqv4\") pod \"kuadrant-operator-catalog-vtp2s\" (UID: \"80fcc130-f349-4cf8-81bc-2d11a079fae0\") " pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:13:56.868105 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.868026    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcqv4\" (UniqueName: \"kubernetes.io/projected/80fcc130-f349-4cf8-81bc-2d11a079fae0-kube-api-access-xcqv4\") pod \"kuadrant-operator-catalog-vtp2s\" (UID: \"80fcc130-f349-4cf8-81bc-2d11a079fae0\") " pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:13:56.877748 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.877715    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcqv4\" (UniqueName: \"kubernetes.io/projected/80fcc130-f349-4cf8-81bc-2d11a079fae0-kube-api-access-xcqv4\") pod \"kuadrant-operator-catalog-vtp2s\" (UID: \"80fcc130-f349-4cf8-81bc-2d11a079fae0\") " pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:13:56.945806 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.945768    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb" event={"ID":"2c308d9e-f92e-4808-ac06-b352826f453b","Type":"ContainerStarted","Data":"503484846d70032ee61f7a9e0e65b8c2220bb4e2e96861a29b6a4b7bb9059edc"}
Jul 15 21:13:56.998380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:56.998347    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:13:57.124687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:57.124661    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vtp2s"]
Jul 15 21:13:57.126150 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:13:57.126121    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80fcc130_f349_4cf8_81bc_2d11a079fae0.slice/crio-5e7c97205b6c1ebe34c3805cc70e483cb5943820c7a40ba0aa865f1efa0ea1ad WatchSource:0}: Error finding container 5e7c97205b6c1ebe34c3805cc70e483cb5943820c7a40ba0aa865f1efa0ea1ad: Status 404 returned error can't find the container with id 5e7c97205b6c1ebe34c3805cc70e483cb5943820c7a40ba0aa865f1efa0ea1ad
Jul 15 21:13:57.953685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:57.953643    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s" event={"ID":"80fcc130-f349-4cf8-81bc-2d11a079fae0","Type":"ContainerStarted","Data":"5e7c97205b6c1ebe34c3805cc70e483cb5943820c7a40ba0aa865f1efa0ea1ad"}
Jul 15 21:13:58.958710 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:58.958677    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb" event={"ID":"2c308d9e-f92e-4808-ac06-b352826f453b","Type":"ContainerStarted","Data":"d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593"}
Jul 15 21:13:58.959211 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:58.958742    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb" podUID="2c308d9e-f92e-4808-ac06-b352826f453b" containerName="registry-server" containerID="cri-o://d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593" gracePeriod=2
Jul 15 21:13:58.960267 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:58.960244    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s" event={"ID":"80fcc130-f349-4cf8-81bc-2d11a079fae0","Type":"ContainerStarted","Data":"0fcf3bf7db5e25745f63151ca81306ef349f686c025201d6f6c322c9e8350f44"}
Jul 15 21:13:58.975072 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:58.975028    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb" podStartSLOduration=0.975227899 podStartE2EDuration="2.975011695s" totalImagesPullingTime="1.999783796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:58.97488926 +0000 UTC m=+443.307030373" watchObservedRunningTime="2026-07-15 21:13:58.975011695 +0000 UTC m=+443.307152787"
Jul 15 21:13:58.992319 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:58.992278    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s" podStartSLOduration=1.517689536 podStartE2EDuration="2.992263764s" totalImagesPullingTime="1.474574228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:58.991453218 +0000 UTC m=+443.323594307" watchObservedRunningTime="2026-07-15 21:13:58.992263764 +0000 UTC m=+443.324404854"
Jul 15 21:13:59.204914 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.204891    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:59.290715 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.290650    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-245v6\" (UniqueName: \"kubernetes.io/projected/2c308d9e-f92e-4808-ac06-b352826f453b-kube-api-access-245v6\") pod \"2c308d9e-f92e-4808-ac06-b352826f453b\" (UID: \"2c308d9e-f92e-4808-ac06-b352826f453b\") "
Jul 15 21:13:59.292729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.292707    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c308d9e-f92e-4808-ac06-b352826f453b-kube-api-access-245v6" (OuterVolumeSpecName: "kube-api-access-245v6") pod "2c308d9e-f92e-4808-ac06-b352826f453b" (UID: "2c308d9e-f92e-4808-ac06-b352826f453b"). InnerVolumeSpecName "kube-api-access-245v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:13:59.391413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.391386    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-245v6\" (UniqueName: \"kubernetes.io/projected/2c308d9e-f92e-4808-ac06-b352826f453b-kube-api-access-245v6\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:13:59.970739 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.970707    2555 generic.go:356] "Generic (PLEG): container finished" podID="2c308d9e-f92e-4808-ac06-b352826f453b" containerID="d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593" exitCode=0
Jul 15 21:13:59.971157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.970758    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb"
Jul 15 21:13:59.971157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.970794    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb" event={"ID":"2c308d9e-f92e-4808-ac06-b352826f453b","Type":"ContainerDied","Data":"d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593"}
Jul 15 21:13:59.971157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.970866    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bpjjb" event={"ID":"2c308d9e-f92e-4808-ac06-b352826f453b","Type":"ContainerDied","Data":"503484846d70032ee61f7a9e0e65b8c2220bb4e2e96861a29b6a4b7bb9059edc"}
Jul 15 21:13:59.971157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.970888    2555 scope.go:117] "RemoveContainer" containerID="d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593"
Jul 15 21:13:59.979641 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.979625    2555 scope.go:117] "RemoveContainer" containerID="d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593"
Jul 15 21:13:59.979867 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:13:59.979850    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593\": container with ID starting with d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593 not found: ID does not exist" containerID="d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593"
Jul 15 21:13:59.979913 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.979875    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593"} err="failed to get container status \"d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593\": rpc error: code = NotFound desc = could not find container \"d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593\": container with ID starting with d2419d1a3f3a9a2db698c8a1d3406d7d92ba9efb0f4dbe081e35e29f69f5f593 not found: ID does not exist"
Jul 15 21:13:59.992643 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.992624    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpjjb"]
Jul 15 21:13:59.996946 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:13:59.996928    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bpjjb"]
Jul 15 21:14:00.217210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:00.217184    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c308d9e-f92e-4808-ac06-b352826f453b" path="/var/lib/kubelet/pods/2c308d9e-f92e-4808-ac06-b352826f453b/volumes"
Jul 15 21:14:06.998443 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:06.998410    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:14:06.998878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:06.998478    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:14:07.029922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:07.029897    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:14:08.026392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:08.026362    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vtp2s"
Jul 15 21:14:14.209374 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:14.209338    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:14.265754 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:14.265723    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:17.017632 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:17.017598    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:19.061395 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:19.061364    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:19.121047 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:19.121018    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:25.264051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:25.264018    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:25.365229 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:25.365200    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:27.923430 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.923392    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"]
Jul 15 21:14:27.924113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.923844    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c308d9e-f92e-4808-ac06-b352826f453b" containerName="registry-server"
Jul 15 21:14:27.924113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.923875    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c308d9e-f92e-4808-ac06-b352826f453b" containerName="registry-server"
Jul 15 21:14:27.924113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.923989    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c308d9e-f92e-4808-ac06-b352826f453b" containerName="registry-server"
Jul 15 21:14:27.928667 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.928648    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:27.931536 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.931513    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kgqp2\""
Jul 15 21:14:27.931655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.931540    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 21:14:27.938177 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:27.938156    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"]
Jul 15 21:14:28.021094 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:28.021061    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldpv4\" (UniqueName: \"kubernetes.io/projected/dd4a1644-4a84-4e34-bdbc-17b46a830d25-kube-api-access-ldpv4\") pod \"dns-operator-controller-manager-864dd78c4c-m5bp4\" (UID: \"dd4a1644-4a84-4e34-bdbc-17b46a830d25\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:28.121993 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:28.121929    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldpv4\" (UniqueName: \"kubernetes.io/projected/dd4a1644-4a84-4e34-bdbc-17b46a830d25-kube-api-access-ldpv4\") pod \"dns-operator-controller-manager-864dd78c4c-m5bp4\" (UID: \"dd4a1644-4a84-4e34-bdbc-17b46a830d25\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:28.140270 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:28.140244    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldpv4\" (UniqueName: \"kubernetes.io/projected/dd4a1644-4a84-4e34-bdbc-17b46a830d25-kube-api-access-ldpv4\") pod \"dns-operator-controller-manager-864dd78c4c-m5bp4\" (UID: \"dd4a1644-4a84-4e34-bdbc-17b46a830d25\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:28.239584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:28.239521    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:28.377130 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:28.377107    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"]
Jul 15 21:14:28.379817 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:14:28.379784    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4a1644_4a84_4e34_bdbc_17b46a830d25.slice/crio-fb64bf577173b2e16ed3724638ce09dea1ac29d7cb26c589a38854f441a63bc0 WatchSource:0}: Error finding container fb64bf577173b2e16ed3724638ce09dea1ac29d7cb26c589a38854f441a63bc0: Status 404 returned error can't find the container with id fb64bf577173b2e16ed3724638ce09dea1ac29d7cb26c589a38854f441a63bc0
Jul 15 21:14:28.892010 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:28.891978    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:29.093768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:29.093737    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" event={"ID":"dd4a1644-4a84-4e34-bdbc-17b46a830d25","Type":"ContainerStarted","Data":"fb64bf577173b2e16ed3724638ce09dea1ac29d7cb26c589a38854f441a63bc0"}
Jul 15 21:14:31.102901 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:31.102809    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" event={"ID":"dd4a1644-4a84-4e34-bdbc-17b46a830d25","Type":"ContainerStarted","Data":"511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca"}
Jul 15 21:14:31.103373 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:31.102941    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:31.126145 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:31.126101    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" podStartSLOduration=1.85618434 podStartE2EDuration="4.126087227s" totalImagesPullingTime="2.269902887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:14:31.125079757 +0000 UTC m=+475.457220879" watchObservedRunningTime="2026-07-15 21:14:31.126087227 +0000 UTC m=+475.458228316"
Jul 15 21:14:31.903547 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:31.903509    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:33.288877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.288841    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"]
Jul 15 21:14:33.291012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.290996    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.293675 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.293650    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kffhx\""
Jul 15 21:14:33.311199 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.311180    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"]
Jul 15 21:14:33.323671 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.323651    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:33.467020 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.466988    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvq6x\" (UniqueName: \"kubernetes.io/projected/c1ff09f2-7b41-4abb-a165-55a5e572d114-kube-api-access-zvq6x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.467173 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.467097    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1ff09f2-7b41-4abb-a165-55a5e572d114-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.567695 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.567618    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1ff09f2-7b41-4abb-a165-55a5e572d114-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.567846 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.567698    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvq6x\" (UniqueName: \"kubernetes.io/projected/c1ff09f2-7b41-4abb-a165-55a5e572d114-kube-api-access-zvq6x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.568017 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.567996    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1ff09f2-7b41-4abb-a165-55a5e572d114-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.579424 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.579395    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvq6x\" (UniqueName: \"kubernetes.io/projected/c1ff09f2-7b41-4abb-a165-55a5e572d114-kube-api-access-zvq6x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.600832 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.600812    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:33.731287 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:33.731262    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"]
Jul 15 21:14:33.733662 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:14:33.733627    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ff09f2_7b41_4abb_a165_55a5e572d114.slice/crio-197c7b2b4be7df9c05b67d1f721e703b7e9625b1b30fa435e35266013efc9617 WatchSource:0}: Error finding container 197c7b2b4be7df9c05b67d1f721e703b7e9625b1b30fa435e35266013efc9617: Status 404 returned error can't find the container with id 197c7b2b4be7df9c05b67d1f721e703b7e9625b1b30fa435e35266013efc9617
Jul 15 21:14:34.121243 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:34.121211    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn" event={"ID":"c1ff09f2-7b41-4abb-a165-55a5e572d114","Type":"ContainerStarted","Data":"197c7b2b4be7df9c05b67d1f721e703b7e9625b1b30fa435e35266013efc9617"}
Jul 15 21:14:34.283459 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:34.283423    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:36.284044 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.284018    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"]
Jul 15 21:14:36.287970 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.287934    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:36.290584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.290565    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mgckh\""
Jul 15 21:14:36.298759 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.298737    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"]
Jul 15 21:14:36.390320 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.390294    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr4s8\" (UniqueName: \"kubernetes.io/projected/7e1d26f4-9900-45ab-b8b2-04f5cad22338-kube-api-access-gr4s8\") pod \"limitador-operator-controller-manager-859d7b78bd-h49fl\" (UID: \"7e1d26f4-9900-45ab-b8b2-04f5cad22338\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:36.478244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.478206    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:36.491578 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.491545    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr4s8\" (UniqueName: \"kubernetes.io/projected/7e1d26f4-9900-45ab-b8b2-04f5cad22338-kube-api-access-gr4s8\") pod \"limitador-operator-controller-manager-859d7b78bd-h49fl\" (UID: \"7e1d26f4-9900-45ab-b8b2-04f5cad22338\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:36.504370 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.504337    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr4s8\" (UniqueName: \"kubernetes.io/projected/7e1d26f4-9900-45ab-b8b2-04f5cad22338-kube-api-access-gr4s8\") pod \"limitador-operator-controller-manager-859d7b78bd-h49fl\" (UID: \"7e1d26f4-9900-45ab-b8b2-04f5cad22338\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:36.575095 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.575068    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:36.597748 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.597723    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:36.674670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.674635    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:36.764811 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.764788    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"]
Jul 15 21:14:36.766727 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:14:36.766690    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1d26f4_9900_45ab_b8b2_04f5cad22338.slice/crio-e702ccd19d078721c59eac80cef5087e085dbbf8c0589a56884626b420b5f39f WatchSource:0}: Error finding container e702ccd19d078721c59eac80cef5087e085dbbf8c0589a56884626b420b5f39f: Status 404 returned error can't find the container with id e702ccd19d078721c59eac80cef5087e085dbbf8c0589a56884626b420b5f39f
Jul 15 21:14:36.772051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.772021    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:36.887169 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:36.887133    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:37.123803 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:37.123772    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:37.136002 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:37.135947    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl" event={"ID":"7e1d26f4-9900-45ab-b8b2-04f5cad22338","Type":"ContainerStarted","Data":"e702ccd19d078721c59eac80cef5087e085dbbf8c0589a56884626b420b5f39f"}
Jul 15 21:14:37.225972 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:37.225932    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:38.845082 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:38.844990    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:39.548740 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.548716    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"]
Jul 15 21:14:39.551832 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.551816    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:39.554790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.554769    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zdwx7\""
Jul 15 21:14:39.563061 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.563030    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"]
Jul 15 21:14:39.616459 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.616434    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngwk6\" (UniqueName: \"kubernetes.io/projected/4295ecb3-f335-4757-8d89-ee38705182d3-kube-api-access-ngwk6\") pod \"authorino-operator-75cc8d5cb7-r9qr5\" (UID: \"4295ecb3-f335-4757-8d89-ee38705182d3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:39.717436 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.717403    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngwk6\" (UniqueName: \"kubernetes.io/projected/4295ecb3-f335-4757-8d89-ee38705182d3-kube-api-access-ngwk6\") pod \"authorino-operator-75cc8d5cb7-r9qr5\" (UID: \"4295ecb3-f335-4757-8d89-ee38705182d3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:39.728286 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.728262    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngwk6\" (UniqueName: \"kubernetes.io/projected/4295ecb3-f335-4757-8d89-ee38705182d3-kube-api-access-ngwk6\") pod \"authorino-operator-75cc8d5cb7-r9qr5\" (UID: \"4295ecb3-f335-4757-8d89-ee38705182d3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:39.898540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.898509    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:39.904331 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:39.904305    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:40.033337 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.033309    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"]
Jul 15 21:14:40.035669 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:14:40.035643    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4295ecb3_f335_4757_8d89_ee38705182d3.slice/crio-bdf7daeae22d0b4a19bd751243c10248b593c27b249ecc48ef250982e75db225 WatchSource:0}: Error finding container bdf7daeae22d0b4a19bd751243c10248b593c27b249ecc48ef250982e75db225: Status 404 returned error can't find the container with id bdf7daeae22d0b4a19bd751243c10248b593c27b249ecc48ef250982e75db225
Jul 15 21:14:40.151600 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.151513    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5" event={"ID":"4295ecb3-f335-4757-8d89-ee38705182d3","Type":"ContainerStarted","Data":"bdf7daeae22d0b4a19bd751243c10248b593c27b249ecc48ef250982e75db225"}
Jul 15 21:14:40.152966 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.152932    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl" event={"ID":"7e1d26f4-9900-45ab-b8b2-04f5cad22338","Type":"ContainerStarted","Data":"1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a"}
Jul 15 21:14:40.153113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.153000    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:40.154491 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.154471    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn" event={"ID":"c1ff09f2-7b41-4abb-a165-55a5e572d114","Type":"ContainerStarted","Data":"9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54"}
Jul 15 21:14:40.154603 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.154595    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:40.172480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.172438    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl" podStartSLOduration=1.4098509799999999 podStartE2EDuration="4.172426189s" totalImagesPullingTime="2.762575209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:14:40.171311921 +0000 UTC m=+484.503453011" watchObservedRunningTime="2026-07-15 21:14:40.172426189 +0000 UTC m=+484.504567278"
Jul 15 21:14:40.194992 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:40.194941    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn" podStartSLOduration=1.3879823230000001 podStartE2EDuration="7.194928647s" totalImagesPullingTime="5.806946324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:14:40.192981882 +0000 UTC m=+484.525122989" watchObservedRunningTime="2026-07-15 21:14:40.194928647 +0000 UTC m=+484.527069736"
Jul 15 21:14:41.310649 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:41.310604    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:41.408580 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:41.408538    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:42.109380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.109349    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:14:42.840697 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.840662    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"]
Jul 15 21:14:42.843248 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.843219    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:42.844022 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.844007    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:42.847100 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.847080    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 21:14:42.847520 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.847505    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cc9h7\""
Jul 15 21:14:42.847723 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.847701    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 21:14:42.854794 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.854774    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"]
Jul 15 21:14:42.943426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.943385    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:42.943426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.943426    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lf7v\" (UniqueName: \"kubernetes.io/projected/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-kube-api-access-7lf7v\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:42.943640 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:42.943504    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.039147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.039112    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:43.044314 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.044290    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.044457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.044365    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.044457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.044390    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lf7v\" (UniqueName: \"kubernetes.io/projected/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-kube-api-access-7lf7v\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.045150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.045127    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.046842 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.046822    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.055608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.055584    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lf7v\" (UniqueName: \"kubernetes.io/projected/e1601410-2ba4-4dbe-9a3c-220b25bc0d05-kube-api-access-7lf7v\") pod \"kuadrant-console-plugin-66b5774985-2g7rk\" (UID: \"e1601410-2ba4-4dbe-9a3c-220b25bc0d05\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.153048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.153025    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"
Jul 15 21:14:43.169691 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.169663    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5" event={"ID":"4295ecb3-f335-4757-8d89-ee38705182d3","Type":"ContainerStarted","Data":"c6f7d76645f6c617a2b93eda825841a5ddb45e19e3afd9a72f7e4f3a45660576"}
Jul 15 21:14:43.169803 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.169787    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:43.194854 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.193974    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5" podStartSLOduration=2.016074242 podStartE2EDuration="4.193909616s" totalImagesPullingTime="2.177835374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:14:43.193174735 +0000 UTC m=+487.525315826" watchObservedRunningTime="2026-07-15 21:14:43.193909616 +0000 UTC m=+487.526050709"
Jul 15 21:14:43.296712 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:43.296413    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk"]
Jul 15 21:14:43.299701 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:14:43.299674    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1601410_2ba4_4dbe_9a3c_220b25bc0d05.slice/crio-8a3e9dcab00babd46b9f351c51828427ab5a111c951a97d7cbce477cf50d6057 WatchSource:0}: Error finding container 8a3e9dcab00babd46b9f351c51828427ab5a111c951a97d7cbce477cf50d6057: Status 404 returned error can't find the container with id 8a3e9dcab00babd46b9f351c51828427ab5a111c951a97d7cbce477cf50d6057
Jul 15 21:14:44.175453 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:44.175420    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk" event={"ID":"e1601410-2ba4-4dbe-9a3c-220b25bc0d05","Type":"ContainerStarted","Data":"8a3e9dcab00babd46b9f351c51828427ab5a111c951a97d7cbce477cf50d6057"}
Jul 15 21:14:44.892585 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:44.892553    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:44.993722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:44.993679    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:51.161997 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:51.161943    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:51.162532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:51.162054    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:52.482550 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.482486    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"]
Jul 15 21:14:52.496123 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.496098    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.501159 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.501126    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"]
Jul 15 21:14:52.531975 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.531920    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjpd\" (UniqueName: \"kubernetes.io/projected/0d60674e-5537-4e3a-96de-32fdd3e5c0de-kube-api-access-kwjpd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.532129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.532073    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d60674e-5537-4e3a-96de-32fdd3e5c0de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.633052 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.633018    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d60674e-5537-4e3a-96de-32fdd3e5c0de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.633224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.633081    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjpd\" (UniqueName: \"kubernetes.io/projected/0d60674e-5537-4e3a-96de-32fdd3e5c0de-kube-api-access-kwjpd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.633525 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.633489    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d60674e-5537-4e3a-96de-32fdd3e5c0de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.648067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.648037    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjpd\" (UniqueName: \"kubernetes.io/projected/0d60674e-5537-4e3a-96de-32fdd3e5c0de-kube-api-access-kwjpd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.649909 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.649884    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"]
Jul 15 21:14:52.650241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.650226    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:14:52.691380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.691343    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"]
Jul 15 21:14:52.697193 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.697169    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:52.721973 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.713420    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"]
Jul 15 21:14:52.736110 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.735275    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:52.736110 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.735359    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr695\" (UniqueName: \"kubernetes.io/projected/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-kube-api-access-hr695\") pod \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:52.836937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.836903    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:52.837124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.836983    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr695\" (UniqueName: \"kubernetes.io/projected/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-kube-api-access-hr695\") pod \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:52.837352 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.837332    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:52.849132 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:52.849105    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr695\" (UniqueName: \"kubernetes.io/projected/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-kube-api-access-hr695\") pod \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:53.010452 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.010373    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:14:53.121730 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.121701    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"]
Jul 15 21:14:53.124556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.124526    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"]
Jul 15 21:14:53.139565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.139540    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"]
Jul 15 21:14:53.146302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.146274    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"]
Jul 15 21:14:53.154906 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.154837    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"]
Jul 15 21:14:53.155528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.155140    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.158797 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.158754    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"]
Jul 15 21:14:53.159233 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.158998    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" containerName="manager" containerID="cri-o://9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54" gracePeriod=2
Jul 15 21:14:53.166355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.166088    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"]
Jul 15 21:14:53.166816 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.166542    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"]
Jul 15 21:14:53.179120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.179094    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"]
Jul 15 21:14:53.179529 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.179316    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" containerName="manager" containerID="cri-o://1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a" gracePeriod=2
Jul 15 21:14:53.179529 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.179371    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:53.181661 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.181635    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:53.183981 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.183930    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:53.185484 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.185427    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"]
Jul 15 21:14:53.185886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.185865    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" containerName="manager"
Jul 15 21:14:53.185886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.185887    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" containerName="manager"
Jul 15 21:14:53.186072 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.186028    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" containerName="manager"
Jul 15 21:14:53.190426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.190401    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"]
Jul 15 21:14:53.190534 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.190511    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.204329 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.204302    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"]
Jul 15 21:14:53.211203 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.211179    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"]
Jul 15 21:14:53.211639 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.211618    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" containerName="manager"
Jul 15 21:14:53.211639 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.211639    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" containerName="manager"
Jul 15 21:14:53.211803 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.211743    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" containerName="manager"
Jul 15 21:14:53.216474 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.216199    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:14:53.220666 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.220639    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:53.223989 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.223931    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:53.226976 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.226839    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:53.228725 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.228668    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"]
Jul 15 21:14:53.229163 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:53.229058    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:53.240263 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.240243    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rkpv\" (UniqueName: \"kubernetes.io/projected/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-kube-api-access-4rkpv\") pod \"kuadrant-operator-controller-manager-55467d9f7-7xtwt\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.240354 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.240292    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xrtz\" (UniqueName: \"kubernetes.io/projected/a54c5808-b33d-41f3-9438-6a193dce0a7f-kube-api-access-6xrtz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w7mgb\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.240354 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.240314    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrfsf\" (UniqueName: \"kubernetes.io/projected/45cfec9f-bc04-497a-bc58-388bd4379e9b-kube-api-access-jrfsf\") pod \"limitador-operator-controller-manager-859d7b78bd-tqm4k\" (UID: \"45cfec9f-bc04-497a-bc58-388bd4379e9b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:14:53.240354 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.240347    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-7xtwt\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.240507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.240370    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a54c5808-b33d-41f3-9438-6a193dce0a7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w7mgb\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.341248 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341212    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-7xtwt\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.341427 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341262    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a54c5808-b33d-41f3-9438-6a193dce0a7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w7mgb\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.341427 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341382    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkpv\" (UniqueName: \"kubernetes.io/projected/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-kube-api-access-4rkpv\") pod \"kuadrant-operator-controller-manager-55467d9f7-7xtwt\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.341556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341443    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xrtz\" (UniqueName: \"kubernetes.io/projected/a54c5808-b33d-41f3-9438-6a193dce0a7f-kube-api-access-6xrtz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w7mgb\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.341556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341469    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrfsf\" (UniqueName: \"kubernetes.io/projected/45cfec9f-bc04-497a-bc58-388bd4379e9b-kube-api-access-jrfsf\") pod \"limitador-operator-controller-manager-859d7b78bd-tqm4k\" (UID: \"45cfec9f-bc04-497a-bc58-388bd4379e9b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:14:53.341660 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341635    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-7xtwt\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.341717 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.341693    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a54c5808-b33d-41f3-9438-6a193dce0a7f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w7mgb\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.351760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.351735    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrfsf\" (UniqueName: \"kubernetes.io/projected/45cfec9f-bc04-497a-bc58-388bd4379e9b-kube-api-access-jrfsf\") pod \"limitador-operator-controller-manager-859d7b78bd-tqm4k\" (UID: \"45cfec9f-bc04-497a-bc58-388bd4379e9b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:14:53.354898 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.354874    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rkpv\" (UniqueName: \"kubernetes.io/projected/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-kube-api-access-4rkpv\") pod \"kuadrant-operator-controller-manager-55467d9f7-7xtwt\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.355870 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.355851    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xrtz\" (UniqueName: \"kubernetes.io/projected/a54c5808-b33d-41f3-9438-6a193dce0a7f-kube-api-access-6xrtz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w7mgb\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.471997 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.471939    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:14:53.578011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.577904    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:14:53.586575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.586551    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:14:53.700410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.700378    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:53.805622 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:53.805578    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:54.176162 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:54.176133    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:54.177851 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:54.177830    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:14:54.211181 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:54.211142    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:14:54.213548 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:14:54.213506    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:14:58.476116 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:58.476080    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:58.547870 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:58.547840    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:59.079895 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:59.079857    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:14:59.136863 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:14:59.136836    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:04.552154 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:04.552118    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:04.652999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:04.652942    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:05.045352 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.045325    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:05.048002 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.047974    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:05.058069 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.057656    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:05.060649 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.060528    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:05.062495 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.062467    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:05.093599 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.093572    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"]
Jul 15 21:15:05.095158 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:15:05.095121    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda54c5808_b33d_41f3_9438_6a193dce0a7f.slice/crio-c066cf5998b2f6cd82a1354d458caae9155cf2cff907cab3a08e9c1d58984953 WatchSource:0}: Error finding container c066cf5998b2f6cd82a1354d458caae9155cf2cff907cab3a08e9c1d58984953: Status 404 returned error can't find the container with id c066cf5998b2f6cd82a1354d458caae9155cf2cff907cab3a08e9c1d58984953
Jul 15 21:15:05.154102 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.154077    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1ff09f2-7b41-4abb-a165-55a5e572d114-extensions-socket-volume\") pod \"c1ff09f2-7b41-4abb-a165-55a5e572d114\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") "
Jul 15 21:15:05.154193 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.154141    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr4s8\" (UniqueName: \"kubernetes.io/projected/7e1d26f4-9900-45ab-b8b2-04f5cad22338-kube-api-access-gr4s8\") pod \"7e1d26f4-9900-45ab-b8b2-04f5cad22338\" (UID: \"7e1d26f4-9900-45ab-b8b2-04f5cad22338\") "
Jul 15 21:15:05.154193 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.154174    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvq6x\" (UniqueName: \"kubernetes.io/projected/c1ff09f2-7b41-4abb-a165-55a5e572d114-kube-api-access-zvq6x\") pod \"c1ff09f2-7b41-4abb-a165-55a5e572d114\" (UID: \"c1ff09f2-7b41-4abb-a165-55a5e572d114\") "
Jul 15 21:15:05.154509 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.154488    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ff09f2-7b41-4abb-a165-55a5e572d114-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c1ff09f2-7b41-4abb-a165-55a5e572d114" (UID: "c1ff09f2-7b41-4abb-a165-55a5e572d114"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:15:05.156583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.156563    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ff09f2-7b41-4abb-a165-55a5e572d114-kube-api-access-zvq6x" (OuterVolumeSpecName: "kube-api-access-zvq6x") pod "c1ff09f2-7b41-4abb-a165-55a5e572d114" (UID: "c1ff09f2-7b41-4abb-a165-55a5e572d114"). InnerVolumeSpecName "kube-api-access-zvq6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:05.156665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.156615    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1d26f4-9900-45ab-b8b2-04f5cad22338-kube-api-access-gr4s8" (OuterVolumeSpecName: "kube-api-access-gr4s8") pod "7e1d26f4-9900-45ab-b8b2-04f5cad22338" (UID: "7e1d26f4-9900-45ab-b8b2-04f5cad22338"). InnerVolumeSpecName "kube-api-access-gr4s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:05.255004 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.254946    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr4s8\" (UniqueName: \"kubernetes.io/projected/7e1d26f4-9900-45ab-b8b2-04f5cad22338-kube-api-access-gr4s8\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:05.255004 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.254997    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvq6x\" (UniqueName: \"kubernetes.io/projected/c1ff09f2-7b41-4abb-a165-55a5e572d114-kube-api-access-zvq6x\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:05.255004 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.255009    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c1ff09f2-7b41-4abb-a165-55a5e572d114-extensions-socket-volume\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:05.289857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.289825    2555 generic.go:356] "Generic (PLEG): container finished" podID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" containerID="1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a" exitCode=0
Jul 15 21:15:05.290040 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.289870    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:05.290040 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.289967    2555 scope.go:117] "RemoveContainer" containerID="1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a"
Jul 15 21:15:05.291942 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.291901    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb" event={"ID":"a54c5808-b33d-41f3-9438-6a193dce0a7f","Type":"ContainerStarted","Data":"4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47"}
Jul 15 21:15:05.292072 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.291980    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:15:05.292072 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.292007    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb" event={"ID":"a54c5808-b33d-41f3-9438-6a193dce0a7f","Type":"ContainerStarted","Data":"c066cf5998b2f6cd82a1354d458caae9155cf2cff907cab3a08e9c1d58984953"}
Jul 15 21:15:05.292882 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.292849    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:05.294408 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.294381    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk" event={"ID":"e1601410-2ba4-4dbe-9a3c-220b25bc0d05","Type":"ContainerStarted","Data":"319af4e4e6b589138167ab1361cee6bed98c0123a4862aaf89ede8772ee148f0"}
Jul 15 21:15:05.295469 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.295440    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:05.296117 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.296094    2555 generic.go:356] "Generic (PLEG): container finished" podID="c1ff09f2-7b41-4abb-a165-55a5e572d114" containerID="9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54" exitCode=0
Jul 15 21:15:05.296217 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.296136    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:05.301371 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.301351    2555 scope.go:117] "RemoveContainer" containerID="1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a"
Jul 15 21:15:05.301652 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.301633    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a\": container with ID starting with 1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a not found: ID does not exist" containerID="1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a"
Jul 15 21:15:05.301694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.301660    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a"} err="failed to get container status \"1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a\": rpc error: code = NotFound desc = could not find container \"1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a\": container with ID starting with 1f9ae225f575275b370c16575745489dc7a86c8aeb5c72bcd3665bbf07a2723a not found: ID does not exist"
Jul 15 21:15:05.301694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.301677    2555 scope.go:117] "RemoveContainer" containerID="9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54"
Jul 15 21:15:05.310267 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.310251    2555 scope.go:117] "RemoveContainer" containerID="9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54"
Jul 15 21:15:05.310521 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.310504    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54\": container with ID starting with 9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54 not found: ID does not exist" containerID="9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54"
Jul 15 21:15:05.310572 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.310527    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54"} err="failed to get container status \"9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54\": rpc error: code = NotFound desc = could not find container \"9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54\": container with ID starting with 9ca7ae77839c2b404ec9ff460bc4067f83349e2656624b239c8ecb3231b79d54 not found: ID does not exist"
Jul 15 21:15:05.331248 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.331196    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb" podStartSLOduration=12.331178367 podStartE2EDuration="12.331178367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:05.320695615 +0000 UTC m=+509.652836705" watchObservedRunningTime="2026-07-15 21:15:05.331178367 +0000 UTC m=+509.663319460"
Jul 15 21:15:05.334310 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.334254    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"]
Jul 15 21:15:05.334653 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:15:05.334628    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad623d01_e29a_4fdb_afc0_ec7fe24809e9.slice/crio-a046081b37548122359a908222551fe5b83abb8a015d1bc866d7617f89538481 WatchSource:0}: Error finding container a046081b37548122359a908222551fe5b83abb8a015d1bc866d7617f89538481: Status 404 returned error can't find the container with id a046081b37548122359a908222551fe5b83abb8a015d1bc866d7617f89538481
Jul 15 21:15:05.335153 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:15:05.335136    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45cfec9f_bc04_497a_bc58_388bd4379e9b.slice/crio-a6679147b9da548650e1f28eac3ba8469f5308531dd435b61e1f7f8908c72d59 WatchSource:0}: Error finding container a6679147b9da548650e1f28eac3ba8469f5308531dd435b61e1f7f8908c72d59: Status 404 returned error can't find the container with id a6679147b9da548650e1f28eac3ba8469f5308531dd435b61e1f7f8908c72d59
Jul 15 21:15:05.336302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.336281    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"]
Jul 15 21:15:05.359133 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:05.359094    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2g7rk" podStartSLOduration=1.702172664 podStartE2EDuration="23.359080928s" totalImagesPullingTime="21.656908264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:05.358776537 +0000 UTC m=+509.690917626" watchObservedRunningTime="2026-07-15 21:15:05.359080928 +0000 UTC m=+509.691222017"
Jul 15 21:15:05.360930 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.360911    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:05.363095 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.363074    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:05.367655 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.367628    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:05.370201 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:05.370161    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:06.216262 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:06.216072    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-h49fl\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h49fl"
Jul 15 21:15:06.218388 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:06.218355    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s8xkn\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8xkn"
Jul 15 21:15:06.219515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.219489    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1d26f4-9900-45ab-b8b2-04f5cad22338" path="/var/lib/kubelet/pods/7e1d26f4-9900-45ab-b8b2-04f5cad22338/volumes"
Jul 15 21:15:06.220013 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.219964    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1ff09f2-7b41-4abb-a165-55a5e572d114" path="/var/lib/kubelet/pods/c1ff09f2-7b41-4abb-a165-55a5e572d114/volumes"
Jul 15 21:15:06.303449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.303418    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k" event={"ID":"45cfec9f-bc04-497a-bc58-388bd4379e9b","Type":"ContainerStarted","Data":"1f034fea10d42d7b90fce58d1acd12eee4a26512ae8cba594b5f5c8ed0305aca"}
Jul 15 21:15:06.303620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.303487    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k" event={"ID":"45cfec9f-bc04-497a-bc58-388bd4379e9b","Type":"ContainerStarted","Data":"a6679147b9da548650e1f28eac3ba8469f5308531dd435b61e1f7f8908c72d59"}
Jul 15 21:15:06.303620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.303549    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:15:06.305940 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.305908    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt" event={"ID":"ad623d01-e29a-4fdb-afc0-ec7fe24809e9","Type":"ContainerStarted","Data":"68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10"}
Jul 15 21:15:06.306067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.305979    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt" event={"ID":"ad623d01-e29a-4fdb-afc0-ec7fe24809e9","Type":"ContainerStarted","Data":"a046081b37548122359a908222551fe5b83abb8a015d1bc866d7617f89538481"}
Jul 15 21:15:06.306067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.306018    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:15:06.324940 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:06.324892    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k" podStartSLOduration=13.324879169 podStartE2EDuration="13.324879169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:06.323868326 +0000 UTC m=+510.656009418" watchObservedRunningTime="2026-07-15 21:15:06.324879169 +0000 UTC m=+510.657020258"
Jul 15 21:15:07.459652 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:07.459606    2555 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-fqpd2_kuadrant-system_6fbd9e3d-14bd-436e-ae19-3ded75af50f8_0(514b7af5e51130e2262684bdf8865879e3a8bc5df10e199ee6479d65322e39a5): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-fqpd2 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 15 21:15:07.460014 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:07.459619    2555 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-wpq9d_kuadrant-system_0d60674e-5537-4e3a-96de-32fdd3e5c0de_0(fa6277f61410133cd213c710ae85c2048a22340d1f4df6e4a7094cb236942550): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-wpq9d to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 15 21:15:07.460014 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:07.459687    2555 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-fqpd2_kuadrant-system_6fbd9e3d-14bd-436e-ae19-3ded75af50f8_0(514b7af5e51130e2262684bdf8865879e3a8bc5df10e199ee6479d65322e39a5): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-fqpd2 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:07.460014 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:07.459757    2555 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-wpq9d_kuadrant-system_0d60674e-5537-4e3a-96de-32fdd3e5c0de_0(fa6277f61410133cd213c710ae85c2048a22340d1f4df6e4a7094cb236942550): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-wpq9d to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:07.912203 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:07.912171    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:08.024106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.024074    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:08.315360 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.315281    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:08.315505 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.315281    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:08.317576 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:08.317551    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:08.320598 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:08.320573    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:08.322082 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.322066    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:08.322400 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:08.322379    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:08.324249 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.324232    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:08.324331 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:08.324286    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:08.326188 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:08.326170    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:08.328216 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:08.328197    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:08.382609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.382587    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr695\" (UniqueName: \"kubernetes.io/projected/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-kube-api-access-hr695\") pod \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") "
Jul 15 21:15:08.382698 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.382633    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d60674e-5537-4e3a-96de-32fdd3e5c0de-extensions-socket-volume\") pod \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") "
Jul 15 21:15:08.382698 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.382666    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwjpd\" (UniqueName: \"kubernetes.io/projected/0d60674e-5537-4e3a-96de-32fdd3e5c0de-kube-api-access-kwjpd\") pod \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\" (UID: \"0d60674e-5537-4e3a-96de-32fdd3e5c0de\") "
Jul 15 21:15:08.382802 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.382707    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-extensions-socket-volume\") pod \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\" (UID: \"6fbd9e3d-14bd-436e-ae19-3ded75af50f8\") "
Jul 15 21:15:08.382992 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.382935    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d60674e-5537-4e3a-96de-32fdd3e5c0de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0d60674e-5537-4e3a-96de-32fdd3e5c0de" (UID: "0d60674e-5537-4e3a-96de-32fdd3e5c0de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:15:08.383052 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.383009    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6fbd9e3d-14bd-436e-ae19-3ded75af50f8" (UID: "6fbd9e3d-14bd-436e-ae19-3ded75af50f8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:15:08.384658 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.384637    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-kube-api-access-hr695" (OuterVolumeSpecName: "kube-api-access-hr695") pod "6fbd9e3d-14bd-436e-ae19-3ded75af50f8" (UID: "6fbd9e3d-14bd-436e-ae19-3ded75af50f8"). InnerVolumeSpecName "kube-api-access-hr695". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:08.384709 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.384656    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d60674e-5537-4e3a-96de-32fdd3e5c0de-kube-api-access-kwjpd" (OuterVolumeSpecName: "kube-api-access-kwjpd") pod "0d60674e-5537-4e3a-96de-32fdd3e5c0de" (UID: "0d60674e-5537-4e3a-96de-32fdd3e5c0de"). InnerVolumeSpecName "kube-api-access-kwjpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:08.483495 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.483464    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-extensions-socket-volume\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:08.483495 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.483491    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr695\" (UniqueName: \"kubernetes.io/projected/6fbd9e3d-14bd-436e-ae19-3ded75af50f8-kube-api-access-hr695\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:08.483851 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.483502    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d60674e-5537-4e3a-96de-32fdd3e5c0de-extensions-socket-volume\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:08.483851 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:08.483513    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwjpd\" (UniqueName: \"kubernetes.io/projected/0d60674e-5537-4e3a-96de-32fdd3e5c0de-kube-api-access-kwjpd\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:09.319300 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:09.319269    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:09.319501 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:09.319270    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:09.321751 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:09.321724    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:09.323810 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:09.323790    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:09.326043 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:09.326019    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:09.328144 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:09.328124    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:09.330151 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:09.330131    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fqpd2\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fqpd2"
Jul 15 21:15:09.332070 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:09.332050    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wpq9d\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wpq9d"
Jul 15 21:15:10.216688 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:10.216655    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d60674e-5537-4e3a-96de-32fdd3e5c0de" path="/var/lib/kubelet/pods/0d60674e-5537-4e3a-96de-32fdd3e5c0de/volumes"
Jul 15 21:15:10.217055 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:10.216890    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fbd9e3d-14bd-436e-ae19-3ded75af50f8" path="/var/lib/kubelet/pods/6fbd9e3d-14bd-436e-ae19-3ded75af50f8/volumes"
Jul 15 21:15:10.247886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:10.247846    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:16.087528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:16.087495    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:16.187807 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:16.187779    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:16.308559 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:16.308528    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:15:16.329918 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:16.329869    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt" podStartSLOduration=23.32985521 podStartE2EDuration="23.32985521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:06.348443472 +0000 UTC m=+510.680584614" watchObservedRunningTime="2026-07-15 21:15:16.32985521 +0000 UTC m=+520.661996300"
Jul 15 21:15:17.312191 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.312165    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:15:17.312583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.312515    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:15:17.382148 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.382118    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"]
Jul 15 21:15:17.382363 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.382342    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb" podUID="a54c5808-b33d-41f3-9438-6a193dce0a7f" containerName="manager" containerID="cri-o://4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47" gracePeriod=10
Jul 15 21:15:17.625565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.625536    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:15:17.684721 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.684690    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"]
Jul 15 21:15:17.685073 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.685058    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a54c5808-b33d-41f3-9438-6a193dce0a7f" containerName="manager"
Jul 15 21:15:17.685073 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.685074    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a54c5808-b33d-41f3-9438-6a193dce0a7f" containerName="manager"
Jul 15 21:15:17.685163 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.685143    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a54c5808-b33d-41f3-9438-6a193dce0a7f" containerName="manager"
Jul 15 21:15:17.687009 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.686990    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.702129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.702095    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"]
Jul 15 21:15:17.760912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.760887    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xrtz\" (UniqueName: \"kubernetes.io/projected/a54c5808-b33d-41f3-9438-6a193dce0a7f-kube-api-access-6xrtz\") pod \"a54c5808-b33d-41f3-9438-6a193dce0a7f\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") "
Jul 15 21:15:17.761067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.760917    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a54c5808-b33d-41f3-9438-6a193dce0a7f-extensions-socket-volume\") pod \"a54c5808-b33d-41f3-9438-6a193dce0a7f\" (UID: \"a54c5808-b33d-41f3-9438-6a193dce0a7f\") "
Jul 15 21:15:17.761188 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.761166    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38736c95-869c-48f6-83c1-de35351d9b81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qpxcm\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.761266 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.761205    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvdkj\" (UniqueName: \"kubernetes.io/projected/38736c95-869c-48f6-83c1-de35351d9b81-kube-api-access-gvdkj\") pod \"kuadrant-operator-controller-manager-86878c769f-qpxcm\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.761349 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.761332    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a54c5808-b33d-41f3-9438-6a193dce0a7f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a54c5808-b33d-41f3-9438-6a193dce0a7f" (UID: "a54c5808-b33d-41f3-9438-6a193dce0a7f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:15:17.762934 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.762912    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a54c5808-b33d-41f3-9438-6a193dce0a7f-kube-api-access-6xrtz" (OuterVolumeSpecName: "kube-api-access-6xrtz") pod "a54c5808-b33d-41f3-9438-6a193dce0a7f" (UID: "a54c5808-b33d-41f3-9438-6a193dce0a7f"). InnerVolumeSpecName "kube-api-access-6xrtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:17.862608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.862525    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38736c95-869c-48f6-83c1-de35351d9b81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qpxcm\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.862608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.862560    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvdkj\" (UniqueName: \"kubernetes.io/projected/38736c95-869c-48f6-83c1-de35351d9b81-kube-api-access-gvdkj\") pod \"kuadrant-operator-controller-manager-86878c769f-qpxcm\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.862773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.862631    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xrtz\" (UniqueName: \"kubernetes.io/projected/a54c5808-b33d-41f3-9438-6a193dce0a7f-kube-api-access-6xrtz\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:17.862773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.862641    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a54c5808-b33d-41f3-9438-6a193dce0a7f-extensions-socket-volume\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:17.862887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.862857    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38736c95-869c-48f6-83c1-de35351d9b81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qpxcm\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.872109 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.872080    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvdkj\" (UniqueName: \"kubernetes.io/projected/38736c95-869c-48f6-83c1-de35351d9b81-kube-api-access-gvdkj\") pod \"kuadrant-operator-controller-manager-86878c769f-qpxcm\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:17.996194 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:17.996174    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:18.131209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.131185    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"]
Jul 15 21:15:18.132773 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:15:18.132750    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38736c95_869c_48f6_83c1_de35351d9b81.slice/crio-15fb0e45f4e238a8c3a0d8e44c54c2b3fcbb76aa825f208bd54ffa3992ac8115 WatchSource:0}: Error finding container 15fb0e45f4e238a8c3a0d8e44c54c2b3fcbb76aa825f208bd54ffa3992ac8115: Status 404 returned error can't find the container with id 15fb0e45f4e238a8c3a0d8e44c54c2b3fcbb76aa825f208bd54ffa3992ac8115
Jul 15 21:15:18.370114 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.370086    2555 generic.go:356] "Generic (PLEG): container finished" podID="a54c5808-b33d-41f3-9438-6a193dce0a7f" containerID="4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47" exitCode=0
Jul 15 21:15:18.370503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.370129    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"
Jul 15 21:15:18.370503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.370177    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb" event={"ID":"a54c5808-b33d-41f3-9438-6a193dce0a7f","Type":"ContainerDied","Data":"4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47"}
Jul 15 21:15:18.370503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.370229    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb" event={"ID":"a54c5808-b33d-41f3-9438-6a193dce0a7f","Type":"ContainerDied","Data":"c066cf5998b2f6cd82a1354d458caae9155cf2cff907cab3a08e9c1d58984953"}
Jul 15 21:15:18.370503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.370256    2555 scope.go:117] "RemoveContainer" containerID="4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47"
Jul 15 21:15:18.372025 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.372002    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" event={"ID":"38736c95-869c-48f6-83c1-de35351d9b81","Type":"ContainerStarted","Data":"287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"}
Jul 15 21:15:18.372121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.372049    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" event={"ID":"38736c95-869c-48f6-83c1-de35351d9b81","Type":"ContainerStarted","Data":"15fb0e45f4e238a8c3a0d8e44c54c2b3fcbb76aa825f208bd54ffa3992ac8115"}
Jul 15 21:15:18.372198 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.372184    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:18.378800 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.378785    2555 scope.go:117] "RemoveContainer" containerID="4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47"
Jul 15 21:15:18.379068 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:18.379049    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47\": container with ID starting with 4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47 not found: ID does not exist" containerID="4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47"
Jul 15 21:15:18.379118 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.379077    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47"} err="failed to get container status \"4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47\": rpc error: code = NotFound desc = could not find container \"4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47\": container with ID starting with 4ca5b8f292abab88acf3c11d8b30f5adeb576a4fbb146af0f442797f34234f47 not found: ID does not exist"
Jul 15 21:15:18.395334 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.395254    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" podStartSLOduration=1.395243171 podStartE2EDuration="1.395243171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:18.394439535 +0000 UTC m=+522.726580655" watchObservedRunningTime="2026-07-15 21:15:18.395243171 +0000 UTC m=+522.727384261"
Jul 15 21:15:18.411621 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.411598    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"]
Jul 15 21:15:18.416057 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:18.416034    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w7mgb"]
Jul 15 21:15:20.216509 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:20.216482    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a54c5808-b33d-41f3-9438-6a193dce0a7f" path="/var/lib/kubelet/pods/a54c5808-b33d-41f3-9438-6a193dce0a7f/volumes"
Jul 15 21:15:20.307046 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:20.307004    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:20.400357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:20.400330    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:22.910381 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:22.910343    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:23.008886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:23.008850    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:26.718377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:26.718342    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:26.819709 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:26.819676    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:29.380687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.380604    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:15:29.438658 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.438622    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"]
Jul 15 21:15:29.438915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.438889    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt" podUID="ad623d01-e29a-4fdb-afc0-ec7fe24809e9" containerName="manager" containerID="cri-o://68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10" gracePeriod=10
Jul 15 21:15:29.711418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.711396    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:15:29.869190 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.869157    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-extensions-socket-volume\") pod \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") "
Jul 15 21:15:29.869357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.869272    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rkpv\" (UniqueName: \"kubernetes.io/projected/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-kube-api-access-4rkpv\") pod \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\" (UID: \"ad623d01-e29a-4fdb-afc0-ec7fe24809e9\") "
Jul 15 21:15:29.869527 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.869505    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ad623d01-e29a-4fdb-afc0-ec7fe24809e9" (UID: "ad623d01-e29a-4fdb-afc0-ec7fe24809e9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:15:29.871271 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.871239    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-kube-api-access-4rkpv" (OuterVolumeSpecName: "kube-api-access-4rkpv") pod "ad623d01-e29a-4fdb-afc0-ec7fe24809e9" (UID: "ad623d01-e29a-4fdb-afc0-ec7fe24809e9"). InnerVolumeSpecName "kube-api-access-4rkpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:29.970665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.970592    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rkpv\" (UniqueName: \"kubernetes.io/projected/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-kube-api-access-4rkpv\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:29.970665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:29.970620    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad623d01-e29a-4fdb-afc0-ec7fe24809e9-extensions-socket-volume\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:15:30.432640 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.432608    2555 generic.go:356] "Generic (PLEG): container finished" podID="ad623d01-e29a-4fdb-afc0-ec7fe24809e9" containerID="68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10" exitCode=0
Jul 15 21:15:30.433026 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.432655    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"
Jul 15 21:15:30.433026 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.432688    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt" event={"ID":"ad623d01-e29a-4fdb-afc0-ec7fe24809e9","Type":"ContainerDied","Data":"68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10"}
Jul 15 21:15:30.433026 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.432745    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt" event={"ID":"ad623d01-e29a-4fdb-afc0-ec7fe24809e9","Type":"ContainerDied","Data":"a046081b37548122359a908222551fe5b83abb8a015d1bc866d7617f89538481"}
Jul 15 21:15:30.433026 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.432773    2555 scope.go:117] "RemoveContainer" containerID="68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10"
Jul 15 21:15:30.440948 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.440805    2555 scope.go:117] "RemoveContainer" containerID="68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10"
Jul 15 21:15:30.441115 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:15:30.441097    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10\": container with ID starting with 68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10 not found: ID does not exist" containerID="68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10"
Jul 15 21:15:30.441158 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.441124    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10"} err="failed to get container status \"68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10\": rpc error: code = NotFound desc = could not find container \"68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10\": container with ID starting with 68123050bf82265559600494f9dbbe77413a3e8d82170771b39deaa4058a9c10 not found: ID does not exist"
Jul 15 21:15:30.459433 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.459412    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"]
Jul 15 21:15:30.467093 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:30.467072    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-7xtwt"]
Jul 15 21:15:32.217113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:32.217087    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad623d01-e29a-4fdb-afc0-ec7fe24809e9" path="/var/lib/kubelet/pods/ad623d01-e29a-4fdb-afc0-ec7fe24809e9/volumes"
Jul 15 21:15:34.033856 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:34.033824    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:34.210121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:34.210071    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:34.908445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:34.908410    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:35.010422 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:35.010384    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:39.705536 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:39.705502    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:39.806916 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:39.806886    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:44.300914 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:44.300881    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:44.358910 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:44.358882    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:45.511726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.511686    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:45.518633 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.518609    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:15:45.519008 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.518994    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad623d01-e29a-4fdb-afc0-ec7fe24809e9" containerName="manager"
Jul 15 21:15:45.519056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.519010    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad623d01-e29a-4fdb-afc0-ec7fe24809e9" containerName="manager"
Jul 15 21:15:45.519090 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.519073    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad623d01-e29a-4fdb-afc0-ec7fe24809e9" containerName="manager"
Jul 15 21:15:45.619234 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.619198    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:15:45.619234 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.619240    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:15:45.619425 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.619311    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.622052 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.622024    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 21:15:45.623999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.623972    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:45.682221 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.682196    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:45.695394 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.695366    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qrfz\" (UniqueName: \"kubernetes.io/projected/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-kube-api-access-2qrfz\") pod \"limitador-limitador-64c94fd64f-bxscp\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.695531 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.695489    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-config-file\") pod \"limitador-limitador-64c94fd64f-bxscp\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.796208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.796115    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-config-file\") pod \"limitador-limitador-64c94fd64f-bxscp\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.796208 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.796152    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qrfz\" (UniqueName: \"kubernetes.io/projected/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-kube-api-access-2qrfz\") pod \"limitador-limitador-64c94fd64f-bxscp\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.796721 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.796699    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-config-file\") pod \"limitador-limitador-64c94fd64f-bxscp\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.807196 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.807171    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qrfz\" (UniqueName: \"kubernetes.io/projected/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-kube-api-access-2qrfz\") pod \"limitador-limitador-64c94fd64f-bxscp\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:45.930665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:45.930628    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:46.062748 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:46.062722    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:15:46.064889 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:15:46.064860    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce10c1b3_4828_4eb8_9f40_d5d495270d5c.slice/crio-71ff52d59381642aab7d6d24332c18b2b9ba8b7a9fd1bc7578b642075c372a58 WatchSource:0}: Error finding container 71ff52d59381642aab7d6d24332c18b2b9ba8b7a9fd1bc7578b642075c372a58: Status 404 returned error can't find the container with id 71ff52d59381642aab7d6d24332c18b2b9ba8b7a9fd1bc7578b642075c372a58
Jul 15 21:15:46.464626 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:46.464592    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:46.507843 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:46.507810    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp" event={"ID":"ce10c1b3-4828-4eb8-9f40-d5d495270d5c","Type":"ContainerStarted","Data":"71ff52d59381642aab7d6d24332c18b2b9ba8b7a9fd1bc7578b642075c372a58"}
Jul 15 21:15:46.565429 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:46.565396    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:15:49.532191 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:49.532156    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp" event={"ID":"ce10c1b3-4828-4eb8-9f40-d5d495270d5c","Type":"ContainerStarted","Data":"f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de"}
Jul 15 21:15:49.532616 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:49.532258    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:15:49.553663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:15:49.553615    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp" podStartSLOduration=1.6490448450000001 podStartE2EDuration="4.553603524s" totalImagesPullingTime="2.904558679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:49.552262888 +0000 UTC m=+553.884403972" watchObservedRunningTime="2026-07-15 21:15:49.553603524 +0000 UTC m=+553.885744626"
Jul 15 21:16:00.537891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:00.537862    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:16:02.833441 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:02.833400    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:16:02.833870 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:02.833711    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp" podUID="ce10c1b3-4828-4eb8-9f40-d5d495270d5c" containerName="limitador" containerID="cri-o://f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de" gracePeriod=30
Jul 15 21:16:03.366289 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.366262    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:16:03.445097 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.445033    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qrfz\" (UniqueName: \"kubernetes.io/projected/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-kube-api-access-2qrfz\") pod \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") "
Jul 15 21:16:03.445097 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.445075    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-config-file\") pod \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\" (UID: \"ce10c1b3-4828-4eb8-9f40-d5d495270d5c\") "
Jul 15 21:16:03.445458 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.445437    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-config-file" (OuterVolumeSpecName: "config-file") pod "ce10c1b3-4828-4eb8-9f40-d5d495270d5c" (UID: "ce10c1b3-4828-4eb8-9f40-d5d495270d5c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:16:03.446922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.446902    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-kube-api-access-2qrfz" (OuterVolumeSpecName: "kube-api-access-2qrfz") pod "ce10c1b3-4828-4eb8-9f40-d5d495270d5c" (UID: "ce10c1b3-4828-4eb8-9f40-d5d495270d5c"). InnerVolumeSpecName "kube-api-access-2qrfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:16:03.545933 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.545896    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qrfz\" (UniqueName: \"kubernetes.io/projected/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-kube-api-access-2qrfz\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:16:03.545933 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.545928    2555 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce10c1b3-4828-4eb8-9f40-d5d495270d5c-config-file\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:16:03.594241 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.594213    2555 generic.go:356] "Generic (PLEG): container finished" podID="ce10c1b3-4828-4eb8-9f40-d5d495270d5c" containerID="f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de" exitCode=0
Jul 15 21:16:03.594355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.594252    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp" event={"ID":"ce10c1b3-4828-4eb8-9f40-d5d495270d5c","Type":"ContainerDied","Data":"f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de"}
Jul 15 21:16:03.594355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.594259    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp"
Jul 15 21:16:03.594355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.594291    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bxscp" event={"ID":"ce10c1b3-4828-4eb8-9f40-d5d495270d5c","Type":"ContainerDied","Data":"71ff52d59381642aab7d6d24332c18b2b9ba8b7a9fd1bc7578b642075c372a58"}
Jul 15 21:16:03.594355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.594313    2555 scope.go:117] "RemoveContainer" containerID="f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de"
Jul 15 21:16:03.602489 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.602472    2555 scope.go:117] "RemoveContainer" containerID="f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de"
Jul 15 21:16:03.602750 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:16:03.602731    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de\": container with ID starting with f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de not found: ID does not exist" containerID="f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de"
Jul 15 21:16:03.602789 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.602760    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de"} err="failed to get container status \"f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de\": rpc error: code = NotFound desc = could not find container \"f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de\": container with ID starting with f6d0d0c5213ac8d218377fe037962b50d9ef896cad206838ce3a93ea42e3c7de not found: ID does not exist"
Jul 15 21:16:03.619539 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.619519    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:16:03.624698 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:03.624677    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bxscp"]
Jul 15 21:16:04.218089 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:04.218062    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce10c1b3-4828-4eb8-9f40-d5d495270d5c" path="/var/lib/kubelet/pods/ce10c1b3-4828-4eb8-9f40-d5d495270d5c/volumes"
Jul 15 21:16:07.494371 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:07.494330    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:07.596618 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:07.596585    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:08.053401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.053373    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ff544"]
Jul 15 21:16:08.053834 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.053806    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce10c1b3-4828-4eb8-9f40-d5d495270d5c" containerName="limitador"
Jul 15 21:16:08.053834 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.053827    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce10c1b3-4828-4eb8-9f40-d5d495270d5c" containerName="limitador"
Jul 15 21:16:08.054057 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.053910    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce10c1b3-4828-4eb8-9f40-d5d495270d5c" containerName="limitador"
Jul 15 21:16:08.056485 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.056465    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.061713 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.061697    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 21:16:08.062825 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.062806    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qj48j\""
Jul 15 21:16:08.062926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.062806    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 21:16:08.062926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.062852    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 21:16:08.073216 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.073195    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ff544"]
Jul 15 21:16:08.103499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.103470    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:08.182556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.182520    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a2f994-c067-42b1-8fff-8c9b3089881e-data\") pod \"postgres-69fb4594cf-ff544\" (UID: \"53a2f994-c067-42b1-8fff-8c9b3089881e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.182714 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.182579    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkh8t\" (UniqueName: \"kubernetes.io/projected/53a2f994-c067-42b1-8fff-8c9b3089881e-kube-api-access-vkh8t\") pod \"postgres-69fb4594cf-ff544\" (UID: \"53a2f994-c067-42b1-8fff-8c9b3089881e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.283759 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.283704    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkh8t\" (UniqueName: \"kubernetes.io/projected/53a2f994-c067-42b1-8fff-8c9b3089881e-kube-api-access-vkh8t\") pod \"postgres-69fb4594cf-ff544\" (UID: \"53a2f994-c067-42b1-8fff-8c9b3089881e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.283998 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.283816    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a2f994-c067-42b1-8fff-8c9b3089881e-data\") pod \"postgres-69fb4594cf-ff544\" (UID: \"53a2f994-c067-42b1-8fff-8c9b3089881e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.284244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.284221    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a2f994-c067-42b1-8fff-8c9b3089881e-data\") pod \"postgres-69fb4594cf-ff544\" (UID: \"53a2f994-c067-42b1-8fff-8c9b3089881e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.294773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.294747    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkh8t\" (UniqueName: \"kubernetes.io/projected/53a2f994-c067-42b1-8fff-8c9b3089881e-kube-api-access-vkh8t\") pod \"postgres-69fb4594cf-ff544\" (UID: \"53a2f994-c067-42b1-8fff-8c9b3089881e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.366380 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.366289    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:08.495560 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.495532    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ff544"]
Jul 15 21:16:08.498050 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:16:08.498022    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a2f994_c067_42b1_8fff_8c9b3089881e.slice/crio-ad733436a660dc880fa4265d959c542b2f60dda21bc2ec3dfd441b6200c3a332 WatchSource:0}: Error finding container ad733436a660dc880fa4265d959c542b2f60dda21bc2ec3dfd441b6200c3a332: Status 404 returned error can't find the container with id ad733436a660dc880fa4265d959c542b2f60dda21bc2ec3dfd441b6200c3a332
Jul 15 21:16:08.617089 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:08.617003    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544" event={"ID":"53a2f994-c067-42b1-8fff-8c9b3089881e","Type":"ContainerStarted","Data":"ad733436a660dc880fa4265d959c542b2f60dda21bc2ec3dfd441b6200c3a332"}
Jul 15 21:16:13.644972 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:13.644927    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544" event={"ID":"53a2f994-c067-42b1-8fff-8c9b3089881e","Type":"ContainerStarted","Data":"ebebfa069e964bae97a90b972c6ba9ad66cd570b8ecb49794359d1b0572464c3"}
Jul 15 21:16:13.645386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:13.645134    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:13.665675 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:13.665625    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544" podStartSLOduration=1.018666857 podStartE2EDuration="5.665609982s" totalImagesPullingTime="4.646943125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:16:13.663354094 +0000 UTC m=+577.995495183" watchObservedRunningTime="2026-07-15 21:16:13.665609982 +0000 UTC m=+577.997751071"
Jul 15 21:16:19.679019 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:19.678990    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ff544"
Jul 15 21:16:20.009475 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.009366    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:20.521265 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.521228    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75c6c7498d-5dkpf"]
Jul 15 21:16:20.527730 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.527707    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:20.530240 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.530213    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75c6c7498d-5dkpf"]
Jul 15 21:16:20.530446 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.530429    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7sq2m\""
Jul 15 21:16:20.679436 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.679398    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:20.697570 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.697546    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5g7\" (UniqueName: \"kubernetes.io/projected/0b563576-c24a-4209-804a-5dc3eb5de399-kube-api-access-xf5g7\") pod \"authorino-75c6c7498d-5dkpf\" (UID: \"0b563576-c24a-4209-804a-5dc3eb5de399\") " pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:20.798791 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.798717    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5g7\" (UniqueName: \"kubernetes.io/projected/0b563576-c24a-4209-804a-5dc3eb5de399-kube-api-access-xf5g7\") pod \"authorino-75c6c7498d-5dkpf\" (UID: \"0b563576-c24a-4209-804a-5dc3eb5de399\") " pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:20.810399 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.810371    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5g7\" (UniqueName: \"kubernetes.io/projected/0b563576-c24a-4209-804a-5dc3eb5de399-kube-api-access-xf5g7\") pod \"authorino-75c6c7498d-5dkpf\" (UID: \"0b563576-c24a-4209-804a-5dc3eb5de399\") " pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:20.840062 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.840038    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:20.966837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:20.966813    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75c6c7498d-5dkpf"]
Jul 15 21:16:20.969411 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:16:20.969386    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b563576_c24a_4209_804a_5dc3eb5de399.slice/crio-8e6ec59650675a5e7e9bd66aa357df3190390df4e54ae1dba14cf61a145d608c WatchSource:0}: Error finding container 8e6ec59650675a5e7e9bd66aa357df3190390df4e54ae1dba14cf61a145d608c: Status 404 returned error can't find the container with id 8e6ec59650675a5e7e9bd66aa357df3190390df4e54ae1dba14cf61a145d608c
Jul 15 21:16:21.359376 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.359342    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75c6c7498d-5dkpf"]
Jul 15 21:16:21.398074 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.398038    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5677cff4cb-x4mss"]
Jul 15 21:16:21.403334 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.403312    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.406066 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.406047    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 21:16:21.409565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.409544    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5677cff4cb-x4mss"]
Jul 15 21:16:21.506121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.505902    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-tls-cert\") pod \"authorino-5677cff4cb-x4mss\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") " pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.506121 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.506003    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbphn\" (UniqueName: \"kubernetes.io/projected/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-kube-api-access-pbphn\") pod \"authorino-5677cff4cb-x4mss\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") " pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.607446 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.607409    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-tls-cert\") pod \"authorino-5677cff4cb-x4mss\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") " pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.607626 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.607507    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbphn\" (UniqueName: \"kubernetes.io/projected/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-kube-api-access-pbphn\") pod \"authorino-5677cff4cb-x4mss\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") " pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.611589 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.611526    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-tls-cert\") pod \"authorino-5677cff4cb-x4mss\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") " pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.620895 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.620865    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbphn\" (UniqueName: \"kubernetes.io/projected/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-kube-api-access-pbphn\") pod \"authorino-5677cff4cb-x4mss\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") " pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.687339 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.687276    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c6c7498d-5dkpf" event={"ID":"0b563576-c24a-4209-804a-5dc3eb5de399","Type":"ContainerStarted","Data":"8e6ec59650675a5e7e9bd66aa357df3190390df4e54ae1dba14cf61a145d608c"}
Jul 15 21:16:21.714207 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.714139    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:16:21.900309 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:21.900220    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5677cff4cb-x4mss"]
Jul 15 21:16:21.903425 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:16:21.903385    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2aee847_1d35_450e_9cd5_f7915e4bfbd7.slice/crio-dd755cd4ff0f24d4c003f076b45ed66713177f0c19e6f87534fa06c3540af735 WatchSource:0}: Error finding container dd755cd4ff0f24d4c003f076b45ed66713177f0c19e6f87534fa06c3540af735: Status 404 returned error can't find the container with id dd755cd4ff0f24d4c003f076b45ed66713177f0c19e6f87534fa06c3540af735
Jul 15 21:16:22.698532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:22.698448    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5677cff4cb-x4mss" event={"ID":"a2aee847-1d35-450e-9cd5-f7915e4bfbd7","Type":"ContainerStarted","Data":"dd755cd4ff0f24d4c003f076b45ed66713177f0c19e6f87534fa06c3540af735"}
Jul 15 21:16:25.713165 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:25.713125    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5677cff4cb-x4mss" event={"ID":"a2aee847-1d35-450e-9cd5-f7915e4bfbd7","Type":"ContainerStarted","Data":"b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb"}
Jul 15 21:16:25.714703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:25.714676    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c6c7498d-5dkpf" event={"ID":"0b563576-c24a-4209-804a-5dc3eb5de399","Type":"ContainerStarted","Data":"b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc"}
Jul 15 21:16:25.714822 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:25.714743    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75c6c7498d-5dkpf" podUID="0b563576-c24a-4209-804a-5dc3eb5de399" containerName="authorino" containerID="cri-o://b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc" gracePeriod=30
Jul 15 21:16:25.733515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:25.733474    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5677cff4cb-x4mss" podStartSLOduration=1.575417271 podStartE2EDuration="4.733459207s" totalImagesPullingTime="3.158041936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:16:25.732396077 +0000 UTC m=+590.064537166" watchObservedRunningTime="2026-07-15 21:16:25.733459207 +0000 UTC m=+590.065600296"
Jul 15 21:16:25.750820 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:25.750771    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75c6c7498d-5dkpf" podStartSLOduration=1.6693284560000001 podStartE2EDuration="5.750753716s" totalImagesPullingTime="4.08142526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:16:25.74901188 +0000 UTC m=+590.081152973" watchObservedRunningTime="2026-07-15 21:16:25.750753716 +0000 UTC m=+590.082894841"
Jul 15 21:16:25.946887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:25.946861    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:26.045197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.045126    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf5g7\" (UniqueName: \"kubernetes.io/projected/0b563576-c24a-4209-804a-5dc3eb5de399-kube-api-access-xf5g7\") pod \"0b563576-c24a-4209-804a-5dc3eb5de399\" (UID: \"0b563576-c24a-4209-804a-5dc3eb5de399\") "
Jul 15 21:16:26.047134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.047109    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b563576-c24a-4209-804a-5dc3eb5de399-kube-api-access-xf5g7" (OuterVolumeSpecName: "kube-api-access-xf5g7") pod "0b563576-c24a-4209-804a-5dc3eb5de399" (UID: "0b563576-c24a-4209-804a-5dc3eb5de399"). InnerVolumeSpecName "kube-api-access-xf5g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:16:26.146206 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.146180    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf5g7\" (UniqueName: \"kubernetes.io/projected/0b563576-c24a-4209-804a-5dc3eb5de399-kube-api-access-xf5g7\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:16:26.720555 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.720521    2555 generic.go:356] "Generic (PLEG): container finished" podID="0b563576-c24a-4209-804a-5dc3eb5de399" containerID="b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc" exitCode=0
Jul 15 21:16:26.721029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.720562    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75c6c7498d-5dkpf"
Jul 15 21:16:26.721029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.720618    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c6c7498d-5dkpf" event={"ID":"0b563576-c24a-4209-804a-5dc3eb5de399","Type":"ContainerDied","Data":"b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc"}
Jul 15 21:16:26.721029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.720663    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75c6c7498d-5dkpf" event={"ID":"0b563576-c24a-4209-804a-5dc3eb5de399","Type":"ContainerDied","Data":"8e6ec59650675a5e7e9bd66aa357df3190390df4e54ae1dba14cf61a145d608c"}
Jul 15 21:16:26.721029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.720685    2555 scope.go:117] "RemoveContainer" containerID="b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc"
Jul 15 21:16:26.728618 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.728603    2555 scope.go:117] "RemoveContainer" containerID="b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc"
Jul 15 21:16:26.728897 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:16:26.728875    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc\": container with ID starting with b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc not found: ID does not exist" containerID="b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc"
Jul 15 21:16:26.729072 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.728902    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc"} err="failed to get container status \"b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc\": rpc error: code = NotFound desc = could not find container \"b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc\": container with ID starting with b039f987894c2f0c78e6b20024690827974f258c9d68d0eb7cc79fde01e0a0cc not found: ID does not exist"
Jul 15 21:16:26.750633 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.750606    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75c6c7498d-5dkpf"]
Jul 15 21:16:26.762582 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:26.762539    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75c6c7498d-5dkpf"]
Jul 15 21:16:28.216544 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:28.216518    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b563576-c24a-4209-804a-5dc3eb5de399" path="/var/lib/kubelet/pods/0b563576-c24a-4209-804a-5dc3eb5de399/volumes"
Jul 15 21:16:30.688903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:30.688867    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:30.845420 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:30.845394    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:30.954802 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:30.954712    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:31.025718 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:31.025689    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:34.038141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:34.038108    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:34.133651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:34.133608    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:36.090496 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:36.090462    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:16:36.091106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:36.091089    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:16:46.562144 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:46.562103    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:16:48.073210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:16:48.073179    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:03.938587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:03.938482    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:04.034209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.034172    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:04.464327 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.464296    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:04.510844 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.510819    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75f458d465-zc65p"]
Jul 15 21:17:04.511168 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.511155    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b563576-c24a-4209-804a-5dc3eb5de399" containerName="authorino"
Jul 15 21:17:04.511218 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.511170    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b563576-c24a-4209-804a-5dc3eb5de399" containerName="authorino"
Jul 15 21:17:04.511253 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.511224    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b563576-c24a-4209-804a-5dc3eb5de399" containerName="authorino"
Jul 15 21:17:04.513286 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.513271    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.516111 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.516090    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 21:17:04.525690 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.525669    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75f458d465-zc65p"]
Jul 15 21:17:04.678625 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.678598    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lxdz\" (UniqueName: \"kubernetes.io/projected/ed662043-f1b8-4c43-9013-37d73ef9c91d-kube-api-access-8lxdz\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.678781 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.678662    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed662043-f1b8-4c43-9013-37d73ef9c91d-openshift-service-ca\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.678781 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.678717    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed662043-f1b8-4c43-9013-37d73ef9c91d-maas-api-tls\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.758053 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.757951    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:04.779114 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.779080    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lxdz\" (UniqueName: \"kubernetes.io/projected/ed662043-f1b8-4c43-9013-37d73ef9c91d-kube-api-access-8lxdz\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.779278 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.779132    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed662043-f1b8-4c43-9013-37d73ef9c91d-openshift-service-ca\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.779278 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.779161    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed662043-f1b8-4c43-9013-37d73ef9c91d-maas-api-tls\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.779854 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.779831    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed662043-f1b8-4c43-9013-37d73ef9c91d-openshift-service-ca\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.781495 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.781467    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed662043-f1b8-4c43-9013-37d73ef9c91d-maas-api-tls\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.791832 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.791807    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lxdz\" (UniqueName: \"kubernetes.io/projected/ed662043-f1b8-4c43-9013-37d73ef9c91d-kube-api-access-8lxdz\") pod \"maas-api-75f458d465-zc65p\" (UID: \"ed662043-f1b8-4c43-9013-37d73ef9c91d\") " pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.823625 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.823597    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:04.959624 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.959601    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75f458d465-zc65p"]
Jul 15 21:17:04.961702 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:17:04.961672    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded662043_f1b8_4c43_9013_37d73ef9c91d.slice/crio-85c026f1d51134c227cb0722b1acca552037e2c675594bc19869c68c54155c08 WatchSource:0}: Error finding container 85c026f1d51134c227cb0722b1acca552037e2c675594bc19869c68c54155c08: Status 404 returned error can't find the container with id 85c026f1d51134c227cb0722b1acca552037e2c675594bc19869c68c54155c08
Jul 15 21:17:04.976298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:04.976277    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:05.072732 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.072660    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:05.113418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.113387    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-svmk4"]
Jul 15 21:17:05.116242 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.116221    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.119386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.119366    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 21:17:05.119511 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.119367    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8h5cb\""
Jul 15 21:17:05.127336 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.127313    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-svmk4"]
Jul 15 21:17:05.283892 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.283857    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9f5ec2c-610b-4946-98d4-429987b4a29d-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-svmk4\" (UID: \"b9f5ec2c-610b-4946-98d4-429987b4a29d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.284106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.283901    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kd5l\" (UniqueName: \"kubernetes.io/projected/b9f5ec2c-610b-4946-98d4-429987b4a29d-kube-api-access-7kd5l\") pod \"payload-processing-f9b8d9b9b-svmk4\" (UID: \"b9f5ec2c-610b-4946-98d4-429987b4a29d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.384377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.384349    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9f5ec2c-610b-4946-98d4-429987b4a29d-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-svmk4\" (UID: \"b9f5ec2c-610b-4946-98d4-429987b4a29d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.384536 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.384384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kd5l\" (UniqueName: \"kubernetes.io/projected/b9f5ec2c-610b-4946-98d4-429987b4a29d-kube-api-access-7kd5l\") pod \"payload-processing-f9b8d9b9b-svmk4\" (UID: \"b9f5ec2c-610b-4946-98d4-429987b4a29d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.385032 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.385014    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9f5ec2c-610b-4946-98d4-429987b4a29d-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-svmk4\" (UID: \"b9f5ec2c-610b-4946-98d4-429987b4a29d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.394440 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.394411    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kd5l\" (UniqueName: \"kubernetes.io/projected/b9f5ec2c-610b-4946-98d4-429987b4a29d-kube-api-access-7kd5l\") pod \"payload-processing-f9b8d9b9b-svmk4\" (UID: \"b9f5ec2c-610b-4946-98d4-429987b4a29d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.429386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.429357    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:05.559804 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.559681    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-svmk4"]
Jul 15 21:17:05.561896 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:17:05.561870    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f5ec2c_610b_4946_98d4_429987b4a29d.slice/crio-257b3992054d59c64ae6e6de6ced7b51d5c2f64d5b245d9bc434c040be1f11fc WatchSource:0}: Error finding container 257b3992054d59c64ae6e6de6ced7b51d5c2f64d5b245d9bc434c040be1f11fc: Status 404 returned error can't find the container with id 257b3992054d59c64ae6e6de6ced7b51d5c2f64d5b245d9bc434c040be1f11fc
Jul 15 21:17:05.899767 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.899731    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p" event={"ID":"ed662043-f1b8-4c43-9013-37d73ef9c91d","Type":"ContainerStarted","Data":"85c026f1d51134c227cb0722b1acca552037e2c675594bc19869c68c54155c08"}
Jul 15 21:17:05.901033 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:05.900987    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4" event={"ID":"b9f5ec2c-610b-4946-98d4-429987b4a29d","Type":"ContainerStarted","Data":"257b3992054d59c64ae6e6de6ced7b51d5c2f64d5b245d9bc434c040be1f11fc"}
Jul 15 21:17:07.912445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:07.912406    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p" event={"ID":"ed662043-f1b8-4c43-9013-37d73ef9c91d","Type":"ContainerStarted","Data":"b503214c8e0096da202a0ea1d03086e587df833619e9da55bfc60741a5826f14"}
Jul 15 21:17:07.912893 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:07.912557    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:07.914057 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:07.914031    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4" event={"ID":"b9f5ec2c-610b-4946-98d4-429987b4a29d","Type":"ContainerStarted","Data":"b8a9197fcdf7e8186d3d4dfb8cac57cdc6efe468d8d26bdfed4675144cedd1f4"}
Jul 15 21:17:07.914157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:07.914090    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:07.934656 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:07.934592    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p" podStartSLOduration=1.19751564 podStartE2EDuration="3.934578821s" totalImagesPullingTime="2.737063181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:17:07.932094392 +0000 UTC m=+632.264235502" watchObservedRunningTime="2026-07-15 21:17:07.934578821 +0000 UTC m=+632.266719908"
Jul 15 21:17:07.959027 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:07.958917    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4" podStartSLOduration=0.824276787 podStartE2EDuration="2.958897298s" totalImagesPullingTime="2.134620511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:17:07.958671582 +0000 UTC m=+632.290812696" watchObservedRunningTime="2026-07-15 21:17:07.958897298 +0000 UTC m=+632.291038389"
Jul 15 21:17:13.924722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:13.924693    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75f458d465-zc65p"
Jul 15 21:17:14.955631 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:14.955584    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:15.056332 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:15.056306    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:18.921061 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:18.921028    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-svmk4"
Jul 15 21:17:28.967692 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:28.967650    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:29.070213 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:29.070180    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:32.564503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:32.564464    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:34.450394 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:34.450362    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:44.653790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:44.653760    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:49.876146 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.876113    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"]
Jul 15 21:17:49.883483 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.883463    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:49.886317 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.886298    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 21:17:49.886439 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.886302    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 21:17:49.887435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.887417    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ms4wp\""
Jul 15 21:17:49.887435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.887434    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 15 21:17:49.890755 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.890730    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"]
Jul 15 21:17:49.920296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.920262    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:17:49.945107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.945084    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:49.945297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.945135    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:49.945297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.945169    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:49.945297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.945193    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:49.945297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.945285    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92f4h\" (UniqueName: \"kubernetes.io/projected/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-kube-api-access-92f4h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:49.945478 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:49.945323    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.046696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.046667    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.046837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.046707    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.046837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.046738    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.046837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.046766    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.046837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.046823    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92f4h\" (UniqueName: \"kubernetes.io/projected/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-kube-api-access-92f4h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.047087 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.046857    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.047165 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.047145    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.047266 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.047238    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.047374 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.047252    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.049012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.048988    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.049283 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.049267    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.056215 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.056196    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92f4h\" (UniqueName: \"kubernetes.io/projected/5207ad27-b704-4b7f-8caa-2dfeeee1ae73-kube-api-access-92f4h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lszcm\" (UID: \"5207ad27-b704-4b7f-8caa-2dfeeee1ae73\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.195880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.195803    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:17:50.327661 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.327637    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"]
Jul 15 21:17:50.329316 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:17:50.329288    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5207ad27_b704_4b7f_8caa_2dfeeee1ae73.slice/crio-64de9f44e7ee3957d19265725c9a4d446ad1fb94838da9c2f6c35528c09a4ee4 WatchSource:0}: Error finding container 64de9f44e7ee3957d19265725c9a4d446ad1fb94838da9c2f6c35528c09a4ee4: Status 404 returned error can't find the container with id 64de9f44e7ee3957d19265725c9a4d446ad1fb94838da9c2f6c35528c09a4ee4
Jul 15 21:17:50.330735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:50.330714    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:17:51.111401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:51.111366    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm" event={"ID":"5207ad27-b704-4b7f-8caa-2dfeeee1ae73","Type":"ContainerStarted","Data":"64de9f44e7ee3957d19265725c9a4d446ad1fb94838da9c2f6c35528c09a4ee4"}
Jul 15 21:17:52.118503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:52.118460    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm" event={"ID":"5207ad27-b704-4b7f-8caa-2dfeeee1ae73","Type":"ContainerStarted","Data":"63008f26110275a388ececeec6aa45cec4ef0e24b7e16e4bd6e25d46049728dc"}
Jul 15 21:17:52.143240 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:52.143185    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm" podStartSLOduration=2.088873974 podStartE2EDuration="3.143170052s" totalImagesPullingTime="1.054296078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:17:52.141447123 +0000 UTC m=+676.473588237" watchObservedRunningTime="2026-07-15 21:17:52.143170052 +0000 UTC m=+676.475311142"
Jul 15 21:17:55.675450 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:17:55.675410    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:18:00.196590 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:00.196549    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:18:00.197011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:00.196623    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:18:00.209098 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:00.209078    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:18:01.171027 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:01.171000    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lszcm"
Jul 15 21:18:03.205878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:03.205844    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:18:09.905657 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:09.905616    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:18:18.296198 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:18.296160    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:18:25.634948 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:18:25.634912    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:19:15.996127 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:15.996084    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:19:16.111409 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:16.111381    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:19:16.940990 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:16.940940    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5677cff4cb-x4mss"]
Jul 15 21:19:16.941188 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:16.941139    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5677cff4cb-x4mss" podUID="a2aee847-1d35-450e-9cd5-f7915e4bfbd7" containerName="authorino" containerID="cri-o://b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb" gracePeriod=30
Jul 15 21:19:17.174239 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.174212    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:19:17.193381 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.193327    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:19:17.343794 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.343763    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:19:17.358918 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.358895    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-tls-cert\") pod \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") "
Jul 15 21:19:17.359075 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.358925    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbphn\" (UniqueName: \"kubernetes.io/projected/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-kube-api-access-pbphn\") pod \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\" (UID: \"a2aee847-1d35-450e-9cd5-f7915e4bfbd7\") "
Jul 15 21:19:17.360890 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.360865    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-kube-api-access-pbphn" (OuterVolumeSpecName: "kube-api-access-pbphn") pod "a2aee847-1d35-450e-9cd5-f7915e4bfbd7" (UID: "a2aee847-1d35-450e-9cd5-f7915e4bfbd7"). InnerVolumeSpecName "kube-api-access-pbphn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:19:17.368384 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.368360    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a2aee847-1d35-450e-9cd5-f7915e4bfbd7" (UID: "a2aee847-1d35-450e-9cd5-f7915e4bfbd7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:19:17.459553 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.459482    2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-tls-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:19:17.459553 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.459506    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbphn\" (UniqueName: \"kubernetes.io/projected/a2aee847-1d35-450e-9cd5-f7915e4bfbd7-kube-api-access-pbphn\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:19:17.507849 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.507814    2555 generic.go:356] "Generic (PLEG): container finished" podID="a2aee847-1d35-450e-9cd5-f7915e4bfbd7" containerID="b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb" exitCode=0
Jul 15 21:19:17.508030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.507860    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5677cff4cb-x4mss"
Jul 15 21:19:17.508030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.507864    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5677cff4cb-x4mss" event={"ID":"a2aee847-1d35-450e-9cd5-f7915e4bfbd7","Type":"ContainerDied","Data":"b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb"}
Jul 15 21:19:17.508030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.507909    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5677cff4cb-x4mss" event={"ID":"a2aee847-1d35-450e-9cd5-f7915e4bfbd7","Type":"ContainerDied","Data":"dd755cd4ff0f24d4c003f076b45ed66713177f0c19e6f87534fa06c3540af735"}
Jul 15 21:19:17.508030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.507926    2555 scope.go:117] "RemoveContainer" containerID="b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb"
Jul 15 21:19:17.517897 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.517876    2555 scope.go:117] "RemoveContainer" containerID="b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb"
Jul 15 21:19:17.518221 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:19:17.518198    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb\": container with ID starting with b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb not found: ID does not exist" containerID="b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb"
Jul 15 21:19:17.518301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.518233    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb"} err="failed to get container status \"b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb\": rpc error: code = NotFound desc = could not find container \"b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb\": container with ID starting with b15a2e177e9771e8bf36d10d7dc2a4b85a2d79cf35d88463a3962101dce93fcb not found: ID does not exist"
Jul 15 21:19:17.552859 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.552831    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5677cff4cb-x4mss"]
Jul 15 21:19:17.565018 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:17.565000    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5677cff4cb-x4mss"]
Jul 15 21:19:18.218264 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:19:18.218232    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2aee847-1d35-450e-9cd5-f7915e4bfbd7" path="/var/lib/kubelet/pods/a2aee847-1d35-450e-9cd5-f7915e4bfbd7/volumes"
Jul 15 21:20:29.787115 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:20:29.787081    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:20:31.183445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:20:31.183416    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:20:50.867711 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:20:50.867672    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:10.709295 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:10.709257    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:10.805136 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:10.805104    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:10.932891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:10.932851    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:17.351371 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:17.351336    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:17.456999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:17.456969    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:17.552584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:17.552544    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:17.710218 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:17.710185    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:23.954564 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:23.954526    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:24.055881 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:24.055848    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:36.124228 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:36.124152    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:21:36.124680 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:36.124556    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:21:38.524164 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:38.524130    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:38.624722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:38.624696    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:21:39.804163 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:21:39.804110    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:24:34.937491 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:24:34.937420    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:25:19.135448 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:19.135418    2555 generic.go:356] "Generic (PLEG): container finished" podID="38736c95-869c-48f6-83c1-de35351d9b81" containerID="287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158" exitCode=0
Jul 15 21:25:19.135869 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:19.135496    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" event={"ID":"38736c95-869c-48f6-83c1-de35351d9b81","Type":"ContainerDied","Data":"287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"}
Jul 15 21:25:19.135869 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:19.135850    2555 scope.go:117] "RemoveContainer" containerID="287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"
Jul 15 21:25:19.379119 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:19.379064    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:25:19.900042 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:19.900002    2555 prober.go:120] "Probe failed" probeType="Liveness" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5" podUID="4295ecb3-f335-4757-8d89-ee38705182d3" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.26:8081/healthz\": dial tcp 10.134.0.26:8081: connect: connection refused"
Jul 15 21:25:20.141327 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.141296    2555 generic.go:356] "Generic (PLEG): container finished" podID="dd4a1644-4a84-4e34-bdbc-17b46a830d25" containerID="511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca" exitCode=1
Jul 15 21:25:20.141713 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.141368    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" event={"ID":"dd4a1644-4a84-4e34-bdbc-17b46a830d25","Type":"ContainerDied","Data":"511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca"}
Jul 15 21:25:20.141769 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.141715    2555 scope.go:117] "RemoveContainer" containerID="511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca"
Jul 15 21:25:20.142945 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.142928    2555 generic.go:356] "Generic (PLEG): container finished" podID="e87f8b62-a84a-479d-b8ea-44ec05e51993" containerID="2294aacf41b289ecab5e2d9c0b001670a65b318b44cc69ef77bd9da754b0cc0f" exitCode=1
Jul 15 21:25:20.143062 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.142978    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24" event={"ID":"e87f8b62-a84a-479d-b8ea-44ec05e51993","Type":"ContainerDied","Data":"2294aacf41b289ecab5e2d9c0b001670a65b318b44cc69ef77bd9da754b0cc0f"}
Jul 15 21:25:20.143248 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.143229    2555 scope.go:117] "RemoveContainer" containerID="2294aacf41b289ecab5e2d9c0b001670a65b318b44cc69ef77bd9da754b0cc0f"
Jul 15 21:25:20.143541 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.143527    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:25:20.144455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.144438    2555 generic.go:356] "Generic (PLEG): container finished" podID="b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89" containerID="5e9a8e3fa9f150546034075574096eb8c4891beca27ed9c81bf77f6273186ea6" exitCode=1
Jul 15 21:25:20.144542 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.144518    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc" event={"ID":"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89","Type":"ContainerDied","Data":"5e9a8e3fa9f150546034075574096eb8c4891beca27ed9c81bf77f6273186ea6"}
Jul 15 21:25:20.144835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.144815    2555 scope.go:117] "RemoveContainer" containerID="5e9a8e3fa9f150546034075574096eb8c4891beca27ed9c81bf77f6273186ea6"
Jul 15 21:25:20.146507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.146476    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" event={"ID":"38736c95-869c-48f6-83c1-de35351d9b81","Type":"ContainerStarted","Data":"0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e"}
Jul 15 21:25:20.146741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.146609    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:25:20.148379 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.148360    2555 generic.go:356] "Generic (PLEG): container finished" podID="42978d06-f6bc-4e7b-9ca4-83606ade276f" containerID="9c7a482fedc45a27780d2ae8d257e050eccbd8217b982760e78d1bc31d1bdd48" exitCode=1
Jul 15 21:25:20.148460 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.148427    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" event={"ID":"42978d06-f6bc-4e7b-9ca4-83606ade276f","Type":"ContainerDied","Data":"9c7a482fedc45a27780d2ae8d257e050eccbd8217b982760e78d1bc31d1bdd48"}
Jul 15 21:25:20.148759 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.148729    2555 scope.go:117] "RemoveContainer" containerID="9c7a482fedc45a27780d2ae8d257e050eccbd8217b982760e78d1bc31d1bdd48"
Jul 15 21:25:20.149896 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.149880    2555 generic.go:356] "Generic (PLEG): container finished" podID="4295ecb3-f335-4757-8d89-ee38705182d3" containerID="c6f7d76645f6c617a2b93eda825841a5ddb45e19e3afd9a72f7e4f3a45660576" exitCode=1
Jul 15 21:25:20.150037 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.149965    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5" event={"ID":"4295ecb3-f335-4757-8d89-ee38705182d3","Type":"ContainerDied","Data":"c6f7d76645f6c617a2b93eda825841a5ddb45e19e3afd9a72f7e4f3a45660576"}
Jul 15 21:25:20.150319 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.150301    2555 scope.go:117] "RemoveContainer" containerID="c6f7d76645f6c617a2b93eda825841a5ddb45e19e3afd9a72f7e4f3a45660576"
Jul 15 21:25:20.151691 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.151676    2555 generic.go:356] "Generic (PLEG): container finished" podID="45cfec9f-bc04-497a-bc58-388bd4379e9b" containerID="1f034fea10d42d7b90fce58d1acd12eee4a26512ae8cba594b5f5c8ed0305aca" exitCode=1
Jul 15 21:25:20.151756 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.151709    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k" event={"ID":"45cfec9f-bc04-497a-bc58-388bd4379e9b","Type":"ContainerDied","Data":"1f034fea10d42d7b90fce58d1acd12eee4a26512ae8cba594b5f5c8ed0305aca"}
Jul 15 21:25:20.151981 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.151966    2555 scope.go:117] "RemoveContainer" containerID="1f034fea10d42d7b90fce58d1acd12eee4a26512ae8cba594b5f5c8ed0305aca"
Jul 15 21:25:20.153448 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.153426    2555 generic.go:356] "Generic (PLEG): container finished" podID="550cae76-7a34-4e8a-bad1-caca322ec031" containerID="8176ee92397379309c8e61a90634ae55f66b2fdfb54f868117d4029db53d0977" exitCode=1
Jul 15 21:25:20.153518 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.153451    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk" event={"ID":"550cae76-7a34-4e8a-bad1-caca322ec031","Type":"ContainerDied","Data":"8176ee92397379309c8e61a90634ae55f66b2fdfb54f868117d4029db53d0977"}
Jul 15 21:25:20.153696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:20.153683    2555 scope.go:117] "RemoveContainer" containerID="8176ee92397379309c8e61a90634ae55f66b2fdfb54f868117d4029db53d0977"
Jul 15 21:25:21.142882 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:25:21.142859    2555 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod dns-operator-controller-manager-864dd78c4c-m5bp4_kuadrant-system(dd4a1644-4a84-4e34-bdbc-17b46a830d25): CreateContainerConfigError: failed to sync configmap cache: timed out waiting for the condition" logger="UnhandledError"
Jul 15 21:25:21.143125 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:25:21.142905    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CreateContainerConfigError: \"failed to sync configmap cache: timed out waiting for the condition\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" podUID="dd4a1644-4a84-4e34-bdbc-17b46a830d25"
Jul 15 21:25:21.159391 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.159364    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc" event={"ID":"b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89","Type":"ContainerStarted","Data":"4dcc9f53f6d43c46853245326eefb425a3a3afc232e7b7bdf1f22fd85fd11ed1"}
Jul 15 21:25:21.159593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.159578    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:25:21.161528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.161497    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74" event={"ID":"42978d06-f6bc-4e7b-9ca4-83606ade276f","Type":"ContainerStarted","Data":"b04a7ed7eaff1e122b3c9ab8a0944c7b80054ed889a12555d662e2878a8d8a41"}
Jul 15 21:25:21.161785 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.161764    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:25:21.163228 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.163209    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5" event={"ID":"4295ecb3-f335-4757-8d89-ee38705182d3","Type":"ContainerStarted","Data":"ebbbd2bcff9c2bce4c7d80fb867fbb1e7342bf571166abd83d0787e49243f9ea"}
Jul 15 21:25:21.163388 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.163374    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:25:21.165011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.164990    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k" event={"ID":"45cfec9f-bc04-497a-bc58-388bd4379e9b","Type":"ContainerStarted","Data":"61ab95d18d1e6f2e1d50350f1b2aac0b43113eeb36de53f51889dfe119a5280a"}
Jul 15 21:25:21.168978 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.165799    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:25:21.174469 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.174437    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk" event={"ID":"550cae76-7a34-4e8a-bad1-caca322ec031","Type":"ContainerStarted","Data":"e6c1e389666876bfa02ce24f870802c0aae258eb20203c7fd55775f4cf1efcd5"}
Jul 15 21:25:21.174669 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:21.174649    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:25:22.107437 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:22.107406    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:25:22.107756 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:22.107742    2555 scope.go:117] "RemoveContainer" containerID="511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca"
Jul 15 21:25:22.107927 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:25:22.107908    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=dns-operator-controller-manager-864dd78c4c-m5bp4_kuadrant-system(dd4a1644-4a84-4e34-bdbc-17b46a830d25)\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" podUID="dd4a1644-4a84-4e34-bdbc-17b46a830d25"
Jul 15 21:25:22.181022 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:22.180978    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24" event={"ID":"e87f8b62-a84a-479d-b8ea-44ec05e51993","Type":"ContainerStarted","Data":"f4515f002989703d72663f2d54b9f8e70364824022dc70a3a1630b97d9d8e2fc"}
Jul 15 21:25:22.181510 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:22.181492    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:25:23.419731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:23.419704    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 21:25:28.240268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:28.240236    2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:25:28.240821 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:28.240669    2555 scope.go:117] "RemoveContainer" containerID="511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca"
Jul 15 21:25:28.240988 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:25:28.240916    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=dns-operator-controller-manager-864dd78c4c-m5bp4_kuadrant-system(dd4a1644-4a84-4e34-bdbc-17b46a830d25)\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" podUID="dd4a1644-4a84-4e34-bdbc-17b46a830d25"
Jul 15 21:25:31.176852 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:31.176822    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:25:32.183217 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:32.183189    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-858d4f578f-rtk74"
Jul 15 21:25:32.183670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:32.183263    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9qr5"
Jul 15 21:25:32.183670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:32.183307    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqm4k"
Jul 15 21:25:33.189537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:33.189507    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j5z24"
Jul 15 21:25:40.212655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:40.212628    2555 scope.go:117] "RemoveContainer" containerID="511acafdd800953c15dfa24466371d567b0afc500287085f012a142ece65c0ca"
Jul 15 21:25:40.274192 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:40.274164    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4" event={"ID":"dd4a1644-4a84-4e34-bdbc-17b46a830d25","Type":"ContainerStarted","Data":"bdbce3a5b51781ca0524fe903a4dc604541035972f28c2f36f5f3815696019ff"}
Jul 15 21:25:40.274482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:40.274465    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:25:51.283940 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:51.283911    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5bp4"
Jul 15 21:25:52.183050 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:52.183018    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5s7kk"
Jul 15 21:25:52.186062 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:25:52.186042    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6pscc"
Jul 15 21:26:04.054311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:04.054220    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"]
Jul 15 21:26:04.054835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:04.054484    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager" containerID="cri-o://0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e" gracePeriod=10
Jul 15 21:26:07.608864 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.608837    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:26:07.697433 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.697359    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvdkj\" (UniqueName: \"kubernetes.io/projected/38736c95-869c-48f6-83c1-de35351d9b81-kube-api-access-gvdkj\") pod \"38736c95-869c-48f6-83c1-de35351d9b81\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") "
Jul 15 21:26:07.697572 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.697468    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38736c95-869c-48f6-83c1-de35351d9b81-extensions-socket-volume\") pod \"38736c95-869c-48f6-83c1-de35351d9b81\" (UID: \"38736c95-869c-48f6-83c1-de35351d9b81\") "
Jul 15 21:26:07.697826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.697799    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38736c95-869c-48f6-83c1-de35351d9b81-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "38736c95-869c-48f6-83c1-de35351d9b81" (UID: "38736c95-869c-48f6-83c1-de35351d9b81"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:26:07.699363 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.699341    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38736c95-869c-48f6-83c1-de35351d9b81-kube-api-access-gvdkj" (OuterVolumeSpecName: "kube-api-access-gvdkj") pod "38736c95-869c-48f6-83c1-de35351d9b81" (UID: "38736c95-869c-48f6-83c1-de35351d9b81"). InnerVolumeSpecName "kube-api-access-gvdkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:26:07.798373 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.798338    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38736c95-869c-48f6-83c1-de35351d9b81-extensions-socket-volume\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:26:07.798373 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:07.798365    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvdkj\" (UniqueName: \"kubernetes.io/projected/38736c95-869c-48f6-83c1-de35351d9b81-kube-api-access-gvdkj\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:26:08.407737 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.407703    2555 generic.go:356] "Generic (PLEG): container finished" podID="38736c95-869c-48f6-83c1-de35351d9b81" containerID="0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e" exitCode=0
Jul 15 21:26:08.407912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.407744    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" event={"ID":"38736c95-869c-48f6-83c1-de35351d9b81","Type":"ContainerDied","Data":"0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e"}
Jul 15 21:26:08.407912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.407801    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm" event={"ID":"38736c95-869c-48f6-83c1-de35351d9b81","Type":"ContainerDied","Data":"15fb0e45f4e238a8c3a0d8e44c54c2b3fcbb76aa825f208bd54ffa3992ac8115"}
Jul 15 21:26:08.407912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.407822    2555 scope.go:117] "RemoveContainer" containerID="0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e"
Jul 15 21:26:08.407912 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.407752    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"
Jul 15 21:26:08.416891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.416869    2555 scope.go:117] "RemoveContainer" containerID="287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"
Jul 15 21:26:08.425418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.425395    2555 scope.go:117] "RemoveContainer" containerID="0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e"
Jul 15 21:26:08.425682 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:26:08.425663    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e\": container with ID starting with 0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e not found: ID does not exist" containerID="0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e"
Jul 15 21:26:08.425733 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.425693    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e"} err="failed to get container status \"0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e\": rpc error: code = NotFound desc = could not find container \"0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e\": container with ID starting with 0c032da731a887ffa2405f52e216a43495389ddfa500c7f0de6441332b06422e not found: ID does not exist"
Jul 15 21:26:08.425733 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.425710    2555 scope.go:117] "RemoveContainer" containerID="287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"
Jul 15 21:26:08.425926 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:26:08.425903    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158\": container with ID starting with 287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158 not found: ID does not exist" containerID="287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"
Jul 15 21:26:08.426012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.425932    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158"} err="failed to get container status \"287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158\": rpc error: code = NotFound desc = could not find container \"287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158\": container with ID starting with 287f4ad7188da57f8f802f4312fe909deac1e27ea921584e73800933fe363158 not found: ID does not exist"
Jul 15 21:26:08.439760 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.439734    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"]
Jul 15 21:26:08.443102 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:08.443082    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpxcm"]
Jul 15 21:26:10.217859 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:10.217819    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38736c95-869c-48f6-83c1-de35351d9b81" path="/var/lib/kubelet/pods/38736c95-869c-48f6-83c1-de35351d9b81/volumes"
Jul 15 21:26:36.155365 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:36.155331    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:26:36.155937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:26:36.155900    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:27:01.744938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:01.744906    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:27:05.286161 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286122    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"]
Jul 15 21:27:05.286538 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286464    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager"
Jul 15 21:27:05.286538 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286475    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager"
Jul 15 21:27:05.286538 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286492    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2aee847-1d35-450e-9cd5-f7915e4bfbd7" containerName="authorino"
Jul 15 21:27:05.286538 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286498    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2aee847-1d35-450e-9cd5-f7915e4bfbd7" containerName="authorino"
Jul 15 21:27:05.286538 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286507    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager"
Jul 15 21:27:05.286538 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286513    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager"
Jul 15 21:27:05.286726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286565    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager"
Jul 15 21:27:05.286726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.286576    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2aee847-1d35-450e-9cd5-f7915e4bfbd7" containerName="authorino"
Jul 15 21:27:05.289445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.289427    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.293389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.293361    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kffhx\""
Jul 15 21:27:05.311388 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.311361    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"]
Jul 15 21:27:05.395637 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.395596    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7zfg\" (UniqueName: \"kubernetes.io/projected/47d03303-830c-42de-93f8-d8348cead365-kube-api-access-b7zfg\") pod \"kuadrant-operator-controller-manager-86878c769f-vnzjp\" (UID: \"47d03303-830c-42de-93f8-d8348cead365\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.395809 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.395649    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d03303-830c-42de-93f8-d8348cead365-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnzjp\" (UID: \"47d03303-830c-42de-93f8-d8348cead365\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.496561 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.496520    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7zfg\" (UniqueName: \"kubernetes.io/projected/47d03303-830c-42de-93f8-d8348cead365-kube-api-access-b7zfg\") pod \"kuadrant-operator-controller-manager-86878c769f-vnzjp\" (UID: \"47d03303-830c-42de-93f8-d8348cead365\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.496561 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.496566    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d03303-830c-42de-93f8-d8348cead365-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnzjp\" (UID: \"47d03303-830c-42de-93f8-d8348cead365\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.496921 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.496905    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47d03303-830c-42de-93f8-d8348cead365-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vnzjp\" (UID: \"47d03303-830c-42de-93f8-d8348cead365\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.509298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.509268    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7zfg\" (UniqueName: \"kubernetes.io/projected/47d03303-830c-42de-93f8-d8348cead365-kube-api-access-b7zfg\") pod \"kuadrant-operator-controller-manager-86878c769f-vnzjp\" (UID: \"47d03303-830c-42de-93f8-d8348cead365\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.599570 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.599482    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:05.751472 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:05.751447    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"]
Jul 15 21:27:05.753505 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:27:05.753481    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d03303_830c_42de_93f8_d8348cead365.slice/crio-c826188ad662d9fb24b96284089053814ec84df1386d535ee21617e565685c54 WatchSource:0}: Error finding container c826188ad662d9fb24b96284089053814ec84df1386d535ee21617e565685c54: Status 404 returned error can't find the container with id c826188ad662d9fb24b96284089053814ec84df1386d535ee21617e565685c54
Jul 15 21:27:06.689996 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:06.689936    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp" event={"ID":"47d03303-830c-42de-93f8-d8348cead365","Type":"ContainerStarted","Data":"ba2dd7eff97429dd31dc7cf95b95e3a683bf48c224e79f9ccd17fb4f0d019fa8"}
Jul 15 21:27:06.690410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:06.690011    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp" event={"ID":"47d03303-830c-42de-93f8-d8348cead365","Type":"ContainerStarted","Data":"c826188ad662d9fb24b96284089053814ec84df1386d535ee21617e565685c54"}
Jul 15 21:27:06.690410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:06.690081    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:06.720277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:06.720217    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp" podStartSLOduration=1.720194191 podStartE2EDuration="1.720194191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:27:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:27:06.718430245 +0000 UTC m=+1231.050571356" watchObservedRunningTime="2026-07-15 21:27:06.720194191 +0000 UTC m=+1231.052335284"
Jul 15 21:27:17.696237 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:17.696206    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vnzjp"
Jul 15 21:27:43.032902 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:43.032815    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:27:43.171711 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:27:43.171677    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:30:00.233131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.233092    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"]
Jul 15 21:30:00.233701 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.233682    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="38736c95-869c-48f6-83c1-de35351d9b81" containerName="manager"
Jul 15 21:30:00.236836 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.236814    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"
Jul 15 21:30:00.240245 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.240225    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4qg\""
Jul 15 21:30:00.247476 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.247460    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"
Jul 15 21:30:00.260237 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.260168    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"]
Jul 15 21:30:00.389857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.389833    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"]
Jul 15 21:30:00.392249 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:30:00.392218    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ffa91ca_5765_41d5_9f07_39779530a32c.slice/crio-6ce5f21d9d334f93c40207a0d144d5d97b537b4429c760cbcaa7acbd7e21a30c WatchSource:0}: Error finding container 6ce5f21d9d334f93c40207a0d144d5d97b537b4429c760cbcaa7acbd7e21a30c: Status 404 returned error can't find the container with id 6ce5f21d9d334f93c40207a0d144d5d97b537b4429c760cbcaa7acbd7e21a30c
Jul 15 21:30:00.512933 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:00.512847    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp" event={"ID":"7ffa91ca-5765-41d5-9f07-39779530a32c","Type":"ContainerStarted","Data":"6ce5f21d9d334f93c40207a0d144d5d97b537b4429c760cbcaa7acbd7e21a30c"}
Jul 15 21:30:04.532877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:04.532845    2555 generic.go:356] "Generic (PLEG): container finished" podID="7ffa91ca-5765-41d5-9f07-39779530a32c" containerID="a671651582ccaa0ca1dc4f7592eeacdc3ebacbff148157d06b8e54a285169529" exitCode=0
Jul 15 21:30:04.533338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:04.532930    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp" event={"ID":"7ffa91ca-5765-41d5-9f07-39779530a32c","Type":"ContainerDied","Data":"a671651582ccaa0ca1dc4f7592eeacdc3ebacbff148157d06b8e54a285169529"}
Jul 15 21:30:05.672033 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:05.672006    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"
Jul 15 21:30:06.543106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:06.543015    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp"
Jul 15 21:30:06.543106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:06.543031    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-txbpp" event={"ID":"7ffa91ca-5765-41d5-9f07-39779530a32c","Type":"ContainerDied","Data":"6ce5f21d9d334f93c40207a0d144d5d97b537b4429c760cbcaa7acbd7e21a30c"}
Jul 15 21:30:06.543106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:06.543074    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ce5f21d9d334f93c40207a0d144d5d97b537b4429c760cbcaa7acbd7e21a30c"
Jul 15 21:30:24.466045 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:24.466013    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:30:53.841492 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:53.841456    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:30:53.942487 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:30:53.942451    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:31:19.747926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:31:19.747891    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:31:20.932098 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:31:20.932063    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:31:20.991882 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:31:20.991848    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:31:30.522184 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:31:30.522150    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:31:36.185189 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:31:36.185161    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:31:36.185853 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:31:36.185832    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:32:04.557190 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:32:04.557114    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:32:56.242083 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:32:56.242048    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:05.973413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:05.973376    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:14.237199 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:14.237165    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:19.341238 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:19.341196    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:22.336133 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:22.336102    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:22.403529 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:22.403498    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:22.881015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:22.880980    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.415367 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.415282    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.478552 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.478516    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.638435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.638408    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.737513 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.737435    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.813673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.813636    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.814122 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.814102    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"]
Jul 15 21:33:29.814464 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.814452    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ffa91ca-5765-41d5-9f07-39779530a32c" containerName="cleanup"
Jul 15 21:33:29.814529 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.814466    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ffa91ca-5765-41d5-9f07-39779530a32c" containerName="cleanup"
Jul 15 21:33:29.814529 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.814526    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ffa91ca-5765-41d5-9f07-39779530a32c" containerName="cleanup"
Jul 15 21:33:29.816660 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.816640    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:29.819341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.819319    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-57e21af7-serving-cert\""
Jul 15 21:33:29.819532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.819503    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:33:29.826726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.826700    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"]
Jul 15 21:33:29.872893 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.872862    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:29.911170 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.911146    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:29.911289 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.911190    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7jk\" (UniqueName: \"kubernetes.io/projected/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-kube-api-access-7z7jk\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:29.911289 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:29.911242    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-openshift-service-ca\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.012514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.012426    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.012514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.012481    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7jk\" (UniqueName: \"kubernetes.io/projected/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-kube-api-access-7z7jk\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.012753 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.012518    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-openshift-service-ca\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.012753 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:33:30.012591    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-serving-cert: secret "maas-api-e2e-ait-57e21af7-serving-cert" not found
Jul 15 21:33:30.012753 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:33:30.012669    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls podName:c5cc5949-fa5a-4510-9b1c-8ed3c144a26c nodeName:}" failed. No retries permitted until 2026-07-15 21:33:30.512647105 +0000 UTC m=+1614.844788177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls") pod "maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" (UID: "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c") : secret "maas-api-e2e-ait-57e21af7-serving-cert" not found
Jul 15 21:33:30.013295 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.013269    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-openshift-service-ca\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.025920 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.025890    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7jk\" (UniqueName: \"kubernetes.io/projected/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-kube-api-access-7z7jk\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.421306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.421268    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:30.517233 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.517203    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:30.517430 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:33:30.517323    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-serving-cert: secret "maas-api-e2e-ait-57e21af7-serving-cert" not found
Jul 15 21:33:30.517430 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:33:30.517393    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls podName:c5cc5949-fa5a-4510-9b1c-8ed3c144a26c nodeName:}" failed. No retries permitted until 2026-07-15 21:33:31.51737365 +0000 UTC m=+1615.849514721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls") pod "maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" (UID: "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c") : secret "maas-api-e2e-ait-57e21af7-serving-cert" not found
Jul 15 21:33:30.525461 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.525434    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:30.626173 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:30.626139    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:31.525243 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:31.525208    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:31.527801 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:31.527769    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") pod \"maas-api-e2e-ait-57e21af7-8dff98f9b-65wml\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:31.629114 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:31.629075    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:31.764646 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:31.764599    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"]
Jul 15 21:33:31.768591 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:31.768568    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:33:32.520666 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:32.520635    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" event={"ID":"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c","Type":"ContainerStarted","Data":"818fe4edc19321180b4f708dc39fdff42b4f14b9bb5f450ce3d55526f6d8a6b6"}
Jul 15 21:33:33.493692 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:33.493659    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:33.527431 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:33.527406    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" event={"ID":"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c","Type":"ContainerStarted","Data":"b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a"}
Jul 15 21:33:33.527573 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:33.527468    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:33.547506 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:33.547454    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" podStartSLOduration=3.846676516 podStartE2EDuration="4.547440411s" totalImagesPullingTime="700.763895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:33:33.546426435 +0000 UTC m=+1617.878567524" watchObservedRunningTime="2026-07-15 21:33:33.547440411 +0000 UTC m=+1617.879581500"
Jul 15 21:33:35.962650 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:35.962615    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:36.069262 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:36.069229    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:38.230153 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:38.230120    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:39.537314 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:39.537287    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:42.189692 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.189657    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"]
Jul 15 21:33:42.190267 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.190210    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" podUID="c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" containerName="maas-api" containerID="cri-o://b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a" gracePeriod=30
Jul 15 21:33:42.218564 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.218532    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:42.434347 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.434326    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:42.523609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.523542    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-openshift-service-ca\") pod \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") "
Jul 15 21:33:42.523609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.523589    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") pod \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") "
Jul 15 21:33:42.523765 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.523626    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z7jk\" (UniqueName: \"kubernetes.io/projected/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-kube-api-access-7z7jk\") pod \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\" (UID: \"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c\") "
Jul 15 21:33:42.523988 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.523938    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" (UID: "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:33:42.525783 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.525749    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" (UID: "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:33:42.525875 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.525795    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-kube-api-access-7z7jk" (OuterVolumeSpecName: "kube-api-access-7z7jk") pod "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" (UID: "c5cc5949-fa5a-4510-9b1c-8ed3c144a26c"). InnerVolumeSpecName "kube-api-access-7z7jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:33:42.576264 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.576227    2555 generic.go:356] "Generic (PLEG): container finished" podID="c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" containerID="b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a" exitCode=0
Jul 15 21:33:42.576415 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.576281    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"
Jul 15 21:33:42.576415 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.576315    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" event={"ID":"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c","Type":"ContainerDied","Data":"b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a"}
Jul 15 21:33:42.576415 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.576388    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml" event={"ID":"c5cc5949-fa5a-4510-9b1c-8ed3c144a26c","Type":"ContainerDied","Data":"818fe4edc19321180b4f708dc39fdff42b4f14b9bb5f450ce3d55526f6d8a6b6"}
Jul 15 21:33:42.576415 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.576411    2555 scope.go:117] "RemoveContainer" containerID="b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a"
Jul 15 21:33:42.585678 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.585658    2555 scope.go:117] "RemoveContainer" containerID="b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a"
Jul 15 21:33:42.585904 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:33:42.585886    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a\": container with ID starting with b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a not found: ID does not exist" containerID="b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a"
Jul 15 21:33:42.586038 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.585912    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a"} err="failed to get container status \"b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a\": rpc error: code = NotFound desc = could not find container \"b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a\": container with ID starting with b2d71724a67db450c19099e233b43b4786ca81b94a49bc2a24e8259c17a89b0a not found: ID does not exist"
Jul 15 21:33:42.599604 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.599582    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"]
Jul 15 21:33:42.607447 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.607422    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-57e21af7-8dff98f9b-65wml"]
Jul 15 21:33:42.625056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.625034    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:33:42.625150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.625061    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:33:42.625150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.625073    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z7jk\" (UniqueName: \"kubernetes.io/projected/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c-kube-api-access-7z7jk\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:33:42.875240 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:42.875208    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:44.217482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:44.217444    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" path="/var/lib/kubelet/pods/c5cc5949-fa5a-4510-9b1c-8ed3c144a26c/volumes"
Jul 15 21:33:49.189594 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.189559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.261975 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.261919    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.514012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.513927    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.609827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.609796    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.674514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.674473    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"]
Jul 15 21:33:49.675356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.675329    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" containerName="maas-api"
Jul 15 21:33:49.675356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.675357    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" containerName="maas-api"
Jul 15 21:33:49.675515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.675502    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5cc5949-fa5a-4510-9b1c-8ed3c144a26c" containerName="maas-api"
Jul 15 21:33:49.680508 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.680481    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.683272 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.683252    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6d9b82ab-gw-openshift-default-dockercfg-9b9p8\""
Jul 15 21:33:49.683535 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.683521    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 21:33:49.690158 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.689850    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"]
Jul 15 21:33:49.723131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.723105    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.783581 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.783514    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.788924 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.788907    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/675f1adc-c0bd-469d-8a2f-238938997820-istio-podinfo\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.788933    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-credential-socket\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.788967    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/675f1adc-c0bd-469d-8a2f-238938997820-istiod-ca-cert\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.789010    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-data\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789186 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.789064    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-istio-token\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789186 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.789121    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-envoy\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789280 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.789191    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-certs\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789280 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.789233    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdmws\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-kube-api-access-kdmws\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.789358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.789279    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-socket\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.889977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-certs\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890021    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdmws\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-kube-api-access-kdmws\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890047    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-socket\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890070    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/675f1adc-c0bd-469d-8a2f-238938997820-istio-podinfo\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890090    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-credential-socket\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890125    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/675f1adc-c0bd-469d-8a2f-238938997820-istiod-ca-cert\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890158    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-data\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890182    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-istio-token\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890257 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890216    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-envoy\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890446    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-certs\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890487    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-socket\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890548    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-credential-socket\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890747 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890701    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-data\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.890878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.890859    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/675f1adc-c0bd-469d-8a2f-238938997820-istiod-ca-cert\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.892400 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.892371    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-envoy\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.892619 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.892600    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/675f1adc-c0bd-469d-8a2f-238938997820-istio-podinfo\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.900093 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.900067    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdmws\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-kube-api-access-kdmws\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.900197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.900160    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-istio-token\") pod \"e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") " pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:49.929618 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.929588    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:49.992616 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:49.992588    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:33:50.036584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.036472    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"]
Jul 15 21:33:50.040847 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.040824    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.044054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.043824    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6d9b82ab-serving-cert\""
Jul 15 21:33:50.044054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.043869    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:33:50.059423 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.059196    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"]
Jul 15 21:33:50.087997 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.087932    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:50.144988 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.144965    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"]
Jul 15 21:33:50.146583 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:33:50.146559    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod675f1adc_c0bd_469d_8a2f_238938997820.slice/crio-4c50ca1a356524863e3353681b4363a639c9edc19acc28224917d455f462d212 WatchSource:0}: Error finding container 4c50ca1a356524863e3353681b4363a639c9edc19acc28224917d455f462d212: Status 404 returned error can't find the container with id 4c50ca1a356524863e3353681b4363a639c9edc19acc28224917d455f462d212
Jul 15 21:33:50.192675 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.192641    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzpqk\" (UniqueName: \"kubernetes.io/projected/6c83cc56-4cfd-476b-891f-1d6d88289a61-kube-api-access-kzpqk\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.193025 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.192704    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c83cc56-4cfd-476b-891f-1d6d88289a61-openshift-service-ca\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.193025 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.192748    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c83cc56-4cfd-476b-891f-1d6d88289a61-maas-api-tls\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.280670 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.280640    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:50.293864 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.293803    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c83cc56-4cfd-476b-891f-1d6d88289a61-openshift-service-ca\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.293864 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.293833    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c83cc56-4cfd-476b-891f-1d6d88289a61-maas-api-tls\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.294092 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.293867    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzpqk\" (UniqueName: \"kubernetes.io/projected/6c83cc56-4cfd-476b-891f-1d6d88289a61-kube-api-access-kzpqk\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.294496 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.294476    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c83cc56-4cfd-476b-891f-1d6d88289a61-openshift-service-ca\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.296350 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.296333    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c83cc56-4cfd-476b-891f-1d6d88289a61-maas-api-tls\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.307086 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.307067    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzpqk\" (UniqueName: \"kubernetes.io/projected/6c83cc56-4cfd-476b-891f-1d6d88289a61-kube-api-access-kzpqk\") pod \"maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.363288 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.363258    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:50.496171 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.496149    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"]
Jul 15 21:33:50.498064 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:33:50.498037    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c83cc56_4cfd_476b_891f_1d6d88289a61.slice/crio-03c3836fc03f98f6b36de97800a9aae14c8769efa6a7b8f6b9b2457369a86d04 WatchSource:0}: Error finding container 03c3836fc03f98f6b36de97800a9aae14c8769efa6a7b8f6b9b2457369a86d04: Status 404 returned error can't find the container with id 03c3836fc03f98f6b36de97800a9aae14c8769efa6a7b8f6b9b2457369a86d04
Jul 15 21:33:50.616731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.616603    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs" event={"ID":"6c83cc56-4cfd-476b-891f-1d6d88289a61","Type":"ContainerStarted","Data":"03c3836fc03f98f6b36de97800a9aae14c8769efa6a7b8f6b9b2457369a86d04"}
Jul 15 21:33:50.618651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.618580    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk" event={"ID":"675f1adc-c0bd-469d-8a2f-238938997820","Type":"ContainerStarted","Data":"4c50ca1a356524863e3353681b4363a639c9edc19acc28224917d455f462d212"}
Jul 15 21:33:50.630302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.630273    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:50.730407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:50.730378    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:51.625640 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:51.625611    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs" event={"ID":"6c83cc56-4cfd-476b-891f-1d6d88289a61","Type":"ContainerStarted","Data":"8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe"}
Jul 15 21:33:51.626042 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:51.625702    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:33:51.645341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:51.645295    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs" podStartSLOduration=1.153779587 podStartE2EDuration="1.645281213s" totalImagesPullingTime="491.501626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:33:51.644259005 +0000 UTC m=+1635.976400120" watchObservedRunningTime="2026-07-15 21:33:51.645281213 +0000 UTC m=+1635.977422302"
Jul 15 21:33:53.719438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:53.719406    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:55.588453 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:55.588419    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"]
Jul 15 21:33:55.605671 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:55.605628    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:55.705107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:55.705078    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:57.511486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:57.511459    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:33:57.637087 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:33:57.637059    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:34:00.728189 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:00.728156    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"]
Jul 15 21:34:00.728625 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:00.728431    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs" podUID="6c83cc56-4cfd-476b-891f-1d6d88289a61" containerName="maas-api" containerID="cri-o://8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe" gracePeriod=30
Jul 15 21:34:00.753183 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:00.753157    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:00.973904 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:00.973881    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:34:01.084891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.084814    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c83cc56-4cfd-476b-891f-1d6d88289a61-openshift-service-ca\") pod \"6c83cc56-4cfd-476b-891f-1d6d88289a61\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") "
Jul 15 21:34:01.084891 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.084858    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzpqk\" (UniqueName: \"kubernetes.io/projected/6c83cc56-4cfd-476b-891f-1d6d88289a61-kube-api-access-kzpqk\") pod \"6c83cc56-4cfd-476b-891f-1d6d88289a61\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") "
Jul 15 21:34:01.085148 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.084913    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c83cc56-4cfd-476b-891f-1d6d88289a61-maas-api-tls\") pod \"6c83cc56-4cfd-476b-891f-1d6d88289a61\" (UID: \"6c83cc56-4cfd-476b-891f-1d6d88289a61\") "
Jul 15 21:34:01.085216 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.085170    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c83cc56-4cfd-476b-891f-1d6d88289a61-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6c83cc56-4cfd-476b-891f-1d6d88289a61" (UID: "6c83cc56-4cfd-476b-891f-1d6d88289a61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:34:01.086989 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.086929    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c83cc56-4cfd-476b-891f-1d6d88289a61-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6c83cc56-4cfd-476b-891f-1d6d88289a61" (UID: "6c83cc56-4cfd-476b-891f-1d6d88289a61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:34:01.087104 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.087013    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c83cc56-4cfd-476b-891f-1d6d88289a61-kube-api-access-kzpqk" (OuterVolumeSpecName: "kube-api-access-kzpqk") pod "6c83cc56-4cfd-476b-891f-1d6d88289a61" (UID: "6c83cc56-4cfd-476b-891f-1d6d88289a61"). InnerVolumeSpecName "kube-api-access-kzpqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:34:01.186358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.186330    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c83cc56-4cfd-476b-891f-1d6d88289a61-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:01.186358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.186353    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c83cc56-4cfd-476b-891f-1d6d88289a61-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:01.186358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.186362    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzpqk\" (UniqueName: \"kubernetes.io/projected/6c83cc56-4cfd-476b-891f-1d6d88289a61-kube-api-access-kzpqk\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:01.681719 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.681685    2555 generic.go:356] "Generic (PLEG): container finished" podID="6c83cc56-4cfd-476b-891f-1d6d88289a61" containerID="8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe" exitCode=0
Jul 15 21:34:01.681892 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.681725    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"
Jul 15 21:34:01.681892 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.681769    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs" event={"ID":"6c83cc56-4cfd-476b-891f-1d6d88289a61","Type":"ContainerDied","Data":"8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe"}
Jul 15 21:34:01.681892 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.681820    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs" event={"ID":"6c83cc56-4cfd-476b-891f-1d6d88289a61","Type":"ContainerDied","Data":"03c3836fc03f98f6b36de97800a9aae14c8769efa6a7b8f6b9b2457369a86d04"}
Jul 15 21:34:01.681892 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.681844    2555 scope.go:117] "RemoveContainer" containerID="8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe"
Jul 15 21:34:01.690511 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.690493    2555 scope.go:117] "RemoveContainer" containerID="8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe"
Jul 15 21:34:01.690770 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:01.690754    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe\": container with ID starting with 8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe not found: ID does not exist" containerID="8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe"
Jul 15 21:34:01.690810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.690779    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe"} err="failed to get container status \"8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe\": rpc error: code = NotFound desc = could not find container \"8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe\": container with ID starting with 8825913b971bd11ae53004b3b4f50293580f3c68091db4b1976c64d2c7f791fe not found: ID does not exist"
Jul 15 21:34:01.704252 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.704228    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"]
Jul 15 21:34:01.712815 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:01.712790    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6d9b82ab-6cdd6c48df-nj4vs"]
Jul 15 21:34:02.217377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:02.217345    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c83cc56-4cfd-476b-891f-1d6d88289a61" path="/var/lib/kubelet/pods/6c83cc56-4cfd-476b-891f-1d6d88289a61/volumes"
Jul 15 21:34:02.287494 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:02.287466    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:05.451429 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:05.451395    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:34:05.451703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:05.451464    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:34:05.451703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:05.451511    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:34:05.706437 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:05.706354    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk" event={"ID":"675f1adc-c0bd-469d-8a2f-238938997820","Type":"ContainerStarted","Data":"e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19"}
Jul 15 21:34:05.706437 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:05.706415    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk" podUID="675f1adc-c0bd-469d-8a2f-238938997820" containerName="istio-proxy" containerID="cri-o://e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19" gracePeriod=30
Jul 15 21:34:05.732298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:05.732249    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk" podStartSLOduration=1.429659057 podStartE2EDuration="16.732234242s" totalImagesPullingTime="15.302575185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:34:05.731304936 +0000 UTC m=+1650.063446053" watchObservedRunningTime="2026-07-15 21:34:05.732234242 +0000 UTC m=+1650.064375331"
Jul 15 21:34:08.858426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.858368    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:08.922238 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.922198    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"]
Jul 15 21:34:08.922752 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.922725    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c83cc56-4cfd-476b-891f-1d6d88289a61" containerName="maas-api"
Jul 15 21:34:08.922752 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.922753    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c83cc56-4cfd-476b-891f-1d6d88289a61" containerName="maas-api"
Jul 15 21:34:08.922894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.922862    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c83cc56-4cfd-476b-891f-1d6d88289a61" containerName="maas-api"
Jul 15 21:34:08.925757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.925737    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.931534 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.931514    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2c26b6eb-openshift-default-dockercfg-b4r7t\""
Jul 15 21:34:08.945432 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.945409    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"]
Jul 15 21:34:08.950673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950641    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-credential-socket\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.950763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950684    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-token\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.950763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950712    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-certs\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.950860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950782    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-envoy\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.950901 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950857    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-podinfo\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.950937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950901    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-socket\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.950937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950927    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2fhc\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-kube-api-access-b2fhc\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.951051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.950976    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-data\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.951108 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.951054    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b24203-7a17-491a-b7f7-0d7ae6434257-istiod-ca-cert\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:08.951861 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:08.951838    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:09.040754 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.040721    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:09.051982 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.051937    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-podinfo\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052108 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052002    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-socket\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052108 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052028    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2fhc\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-kube-api-access-b2fhc\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052108 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052052    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-data\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052108 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052092    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b24203-7a17-491a-b7f7-0d7ae6434257-istiod-ca-cert\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052159    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-credential-socket\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052182    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-token\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052205    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-certs\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052325 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052306    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-envoy\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052523 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052351    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-socket\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052523 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052393    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-data\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052634 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052604    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-certs\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.052691 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052619    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-credential-socket\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.053000 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.052981    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b24203-7a17-491a-b7f7-0d7ae6434257-istiod-ca-cert\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.054664 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.054638    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-envoy\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.054812 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.054792    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-podinfo\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.068327 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.068304    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2fhc\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-kube-api-access-b2fhc\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.071486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.071460    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-token\") pod \"e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") " pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.188476 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.188445    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:09.236578 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.236535    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:09.321443 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.321365    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"]
Jul 15 21:34:09.325140 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.325110    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.330513 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.330215    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:34:09.330785 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.330696    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2c26b6eb-serving-cert\""
Jul 15 21:34:09.341687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.339657    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"]
Jul 15 21:34:09.354598 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.354575    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxpz2\" (UniqueName: \"kubernetes.io/projected/49d51f62-1699-40d6-8714-3e863f78e99c-kube-api-access-mxpz2\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.354825 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.354807    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d51f62-1699-40d6-8714-3e863f78e99c-openshift-service-ca\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.355000 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.354621    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:09.355132 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.355105    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.393356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.393336    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"]
Jul 15 21:34:09.395338 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:34:09.395309    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b24203_7a17_491a_b7f7_0d7ae6434257.slice/crio-7196f6f523268a05f0ff395a2c9a40402f29273ad6da85f1fe4168019f5f883d WatchSource:0}: Error finding container 7196f6f523268a05f0ff395a2c9a40402f29273ad6da85f1fe4168019f5f883d: Status 404 returned error can't find the container with id 7196f6f523268a05f0ff395a2c9a40402f29273ad6da85f1fe4168019f5f883d
Jul 15 21:34:09.398903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.398873    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:34:09.399039 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.398936    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:34:09.399039 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.398979    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:34:09.456428 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.456358    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.456428 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.456407    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxpz2\" (UniqueName: \"kubernetes.io/projected/49d51f62-1699-40d6-8714-3e863f78e99c-kube-api-access-mxpz2\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.456614 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.456436    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d51f62-1699-40d6-8714-3e863f78e99c-openshift-service-ca\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.456614 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:09.456496    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-serving-cert: secret "maas-api-e2e-derive-2c26b6eb-serving-cert" not found
Jul 15 21:34:09.456614 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:09.456554    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls podName:49d51f62-1699-40d6-8714-3e863f78e99c nodeName:}" failed. No retries permitted until 2026-07-15 21:34:09.956538672 +0000 UTC m=+1654.288679743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls") pod "maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" (UID: "49d51f62-1699-40d6-8714-3e863f78e99c") : secret "maas-api-e2e-derive-2c26b6eb-serving-cert" not found
Jul 15 21:34:09.457149 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.457131    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d51f62-1699-40d6-8714-3e863f78e99c-openshift-service-ca\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.469674 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.469652    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxpz2\" (UniqueName: \"kubernetes.io/projected/49d51f62-1699-40d6-8714-3e863f78e99c-kube-api-access-mxpz2\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.728836 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.728744    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8" event={"ID":"f9b24203-7a17-491a-b7f7-0d7ae6434257","Type":"ContainerStarted","Data":"740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23"}
Jul 15 21:34:09.729017 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.728846    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8" event={"ID":"f9b24203-7a17-491a-b7f7-0d7ae6434257","Type":"ContainerStarted","Data":"7196f6f523268a05f0ff395a2c9a40402f29273ad6da85f1fe4168019f5f883d"}
Jul 15 21:34:09.751343 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.751293    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8" podStartSLOduration=1.75127564 podStartE2EDuration="1.75127564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:34:09.74989832 +0000 UTC m=+1654.082039408" watchObservedRunningTime="2026-07-15 21:34:09.75127564 +0000 UTC m=+1654.083416731"
Jul 15 21:34:09.961714 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:09.961674    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:09.962096 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:09.961812    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-serving-cert: secret "maas-api-e2e-derive-2c26b6eb-serving-cert" not found
Jul 15 21:34:09.962096 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:09.961877    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls podName:49d51f62-1699-40d6-8714-3e863f78e99c nodeName:}" failed. No retries permitted until 2026-07-15 21:34:10.961860245 +0000 UTC m=+1655.294001313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls") pod "maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" (UID: "49d51f62-1699-40d6-8714-3e863f78e99c") : secret "maas-api-e2e-derive-2c26b6eb-serving-cert" not found
Jul 15 21:34:10.198239 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.198202    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:10.237429 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.237400    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:10.242663 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.242640    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:10.271540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.271510    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:10.734352 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.734324    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:10.735278 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.735260    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:10.966487 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.966430    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:34:10.970831 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.970811    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:10.972987 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:10.972948    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") pod \"maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:11.071232 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071205    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-istio-token\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071244    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/675f1adc-c0bd-469d-8a2f-238938997820-istio-podinfo\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071275    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdmws\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-kube-api-access-kdmws\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071329    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-data\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071347    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/675f1adc-c0bd-469d-8a2f-238938997820-istiod-ca-cert\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071365    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-envoy\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071396    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-credential-socket\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071438    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-certs\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071463    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-socket\") pod \"675f1adc-c0bd-469d-8a2f-238938997820\" (UID: \"675f1adc-c0bd-469d-8a2f-238938997820\") "
Jul 15 21:34:11.071724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071691    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-data" (OuterVolumeSpecName: "istio-data") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:11.071724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071708    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:11.071943 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.071920    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:11.072159 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.072131    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/675f1adc-c0bd-469d-8a2f-238938997820-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:34:11.072229 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.072200    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:11.073586 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.073559    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/675f1adc-c0bd-469d-8a2f-238938997820-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:34:11.073586 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.073559    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:11.073798 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.073780    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-istio-token" (OuterVolumeSpecName: "istio-token") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:34:11.073798 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.073790    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-kube-api-access-kdmws" (OuterVolumeSpecName: "kube-api-access-kdmws") pod "675f1adc-c0bd-469d-8a2f-238938997820" (UID: "675f1adc-c0bd-469d-8a2f-238938997820"). InnerVolumeSpecName "kube-api-access-kdmws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:34:11.141861 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.141833    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:11.173362 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173338    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173362 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173361    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173369    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173378    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/675f1adc-c0bd-469d-8a2f-238938997820-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173386    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdmws\" (UniqueName: \"kubernetes.io/projected/675f1adc-c0bd-469d-8a2f-238938997820-kube-api-access-kdmws\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173394    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173402    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/675f1adc-c0bd-469d-8a2f-238938997820-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173409    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.173530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.173417    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/675f1adc-c0bd-469d-8a2f-238938997820-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:11.271860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.271835    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"]
Jul 15 21:34:11.273980 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:34:11.273931    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d51f62_1699_40d6_8714_3e863f78e99c.slice/crio-1545f06b7bddec22bf51b68ef53046b08fe347da7568ebf732034f25d3ee5b9c WatchSource:0}: Error finding container 1545f06b7bddec22bf51b68ef53046b08fe347da7568ebf732034f25d3ee5b9c: Status 404 returned error can't find the container with id 1545f06b7bddec22bf51b68ef53046b08fe347da7568ebf732034f25d3ee5b9c
Jul 15 21:34:11.740880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.740844    2555 generic.go:356] "Generic (PLEG): container finished" podID="675f1adc-c0bd-469d-8a2f-238938997820" containerID="e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19" exitCode=0
Jul 15 21:34:11.740880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.740884    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"
Jul 15 21:34:11.741124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.740934    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk" event={"ID":"675f1adc-c0bd-469d-8a2f-238938997820","Type":"ContainerDied","Data":"e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19"}
Jul 15 21:34:11.741124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.740997    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk" event={"ID":"675f1adc-c0bd-469d-8a2f-238938997820","Type":"ContainerDied","Data":"4c50ca1a356524863e3353681b4363a639c9edc19acc28224917d455f462d212"}
Jul 15 21:34:11.741124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.741025    2555 scope.go:117] "RemoveContainer" containerID="e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19"
Jul 15 21:34:11.742251 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.742228    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" event={"ID":"49d51f62-1699-40d6-8714-3e863f78e99c","Type":"ContainerStarted","Data":"1545f06b7bddec22bf51b68ef53046b08fe347da7568ebf732034f25d3ee5b9c"}
Jul 15 21:34:11.750715 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.750695    2555 scope.go:117] "RemoveContainer" containerID="e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19"
Jul 15 21:34:11.750986 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:11.750938    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19\": container with ID starting with e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19 not found: ID does not exist" containerID="e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19"
Jul 15 21:34:11.751066 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.750989    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19"} err="failed to get container status \"e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19\": rpc error: code = NotFound desc = could not find container \"e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19\": container with ID starting with e65cc3740e3e8852a872fd1ebf971a82c51ff6d962127d45974460f2936cae19 not found: ID does not exist"
Jul 15 21:34:11.766486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.766466    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"]
Jul 15 21:34:11.771492 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:11.771472    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6d9b82ab-gw-openshift-default-6cfd96bcff-z9cpk"]
Jul 15 21:34:12.218844 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:12.218806    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="675f1adc-c0bd-469d-8a2f-238938997820" path="/var/lib/kubelet/pods/675f1adc-c0bd-469d-8a2f-238938997820/volumes"
Jul 15 21:34:12.749590 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:12.749562    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" event={"ID":"49d51f62-1699-40d6-8714-3e863f78e99c","Type":"ContainerStarted","Data":"52a74fb4ed94d5da5788e546bf9ea6f72ac6cc21a1681c6f3e5525d8bd9b0ade"}
Jul 15 21:34:12.749876 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:12.749857    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:12.775678 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:12.775637    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" podStartSLOduration=3.262726033 podStartE2EDuration="3.775622261s" totalImagesPullingTime="512.896228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:34:12.773534032 +0000 UTC m=+1657.105675112" watchObservedRunningTime="2026-07-15 21:34:12.775622261 +0000 UTC m=+1657.107763351"
Jul 15 21:34:13.106607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:13.106567    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:14.712279 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:14.712244    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"]
Jul 15 21:34:14.712623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:14.712470    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8" podUID="f9b24203-7a17-491a-b7f7-0d7ae6434257" containerName="istio-proxy" containerID="cri-o://740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23" gracePeriod=30
Jul 15 21:34:14.731981 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:14.731925    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:14.846632 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:14.846600    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:17.063452 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:17.063424    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:18.760721 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:18.760691    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:20.570837 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.570816    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:20.649311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.649231    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"]
Jul 15 21:34:20.649504 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.649472    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" podUID="49d51f62-1699-40d6-8714-3e863f78e99c" containerName="maas-api" containerID="cri-o://52a74fb4ed94d5da5788e546bf9ea6f72ac6cc21a1681c6f3e5525d8bd9b0ade" gracePeriod=30
Jul 15 21:34:20.660356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660333    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-credential-socket\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660485 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660382    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-token\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660485 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660417    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-data\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660485 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660446    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-podinfo\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660485 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660464    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-envoy\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660485 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660485    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b24203-7a17-491a-b7f7-0d7ae6434257-istiod-ca-cert\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660626    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-certs\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660657    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-socket\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660663    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:20.660750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660679    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2fhc\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-kube-api-access-b2fhc\") pod \"f9b24203-7a17-491a-b7f7-0d7ae6434257\" (UID: \"f9b24203-7a17-491a-b7f7-0d7ae6434257\") "
Jul 15 21:34:20.660968 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.660852    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:20.661517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.661066    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:20.661517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.661068    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b24203-7a17-491a-b7f7-0d7ae6434257-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:34:20.661517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.661361    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:20.661517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.661462    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.661517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.661480    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.661517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.661493    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.662937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.662898    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:34:20.663197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.662942    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-kube-api-access-b2fhc" (OuterVolumeSpecName: "kube-api-access-b2fhc") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "kube-api-access-b2fhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:34:20.663197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.663033    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:34:20.663197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.663150    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9b24203-7a17-491a-b7f7-0d7ae6434257" (UID: "f9b24203-7a17-491a-b7f7-0d7ae6434257"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:34:20.671741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.671692    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:20.762053 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.762021    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.762053 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.762053    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.762247 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.762065    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.762247 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.762077    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b24203-7a17-491a-b7f7-0d7ae6434257-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.762247 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.762087    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b24203-7a17-491a-b7f7-0d7ae6434257-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.762247 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.762100    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2fhc\" (UniqueName: \"kubernetes.io/projected/f9b24203-7a17-491a-b7f7-0d7ae6434257-kube-api-access-b2fhc\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.792658 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.792630    2555 generic.go:356] "Generic (PLEG): container finished" podID="49d51f62-1699-40d6-8714-3e863f78e99c" containerID="52a74fb4ed94d5da5788e546bf9ea6f72ac6cc21a1681c6f3e5525d8bd9b0ade" exitCode=0
Jul 15 21:34:20.792868 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.792711    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" event={"ID":"49d51f62-1699-40d6-8714-3e863f78e99c","Type":"ContainerDied","Data":"52a74fb4ed94d5da5788e546bf9ea6f72ac6cc21a1681c6f3e5525d8bd9b0ade"}
Jul 15 21:34:20.794379 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.794295    2555 generic.go:356] "Generic (PLEG): container finished" podID="f9b24203-7a17-491a-b7f7-0d7ae6434257" containerID="740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23" exitCode=0
Jul 15 21:34:20.794379 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.794330    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8" event={"ID":"f9b24203-7a17-491a-b7f7-0d7ae6434257","Type":"ContainerDied","Data":"740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23"}
Jul 15 21:34:20.794379 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.794362    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"
Jul 15 21:34:20.794609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.794394    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8" event={"ID":"f9b24203-7a17-491a-b7f7-0d7ae6434257","Type":"ContainerDied","Data":"7196f6f523268a05f0ff395a2c9a40402f29273ad6da85f1fe4168019f5f883d"}
Jul 15 21:34:20.794609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.794433    2555 scope.go:117] "RemoveContainer" containerID="740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23"
Jul 15 21:34:20.804743 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.804572    2555 scope.go:117] "RemoveContainer" containerID="740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23"
Jul 15 21:34:20.805064 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:34:20.804994    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23\": container with ID starting with 740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23 not found: ID does not exist" containerID="740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23"
Jul 15 21:34:20.805064 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.805034    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23"} err="failed to get container status \"740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23\": rpc error: code = NotFound desc = could not find container \"740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23\": container with ID starting with 740e8960183f687f873c6e07a3fb78841b1f78aac18af0a76c1234ed3e63af23 not found: ID does not exist"
Jul 15 21:34:20.822413 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.822389    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"]
Jul 15 21:34:20.829015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.828944    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2c26b6eb-openshift-default-867d9564f9-5qkg8"]
Jul 15 21:34:20.898460 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.898437    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:20.963724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.963648    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxpz2\" (UniqueName: \"kubernetes.io/projected/49d51f62-1699-40d6-8714-3e863f78e99c-kube-api-access-mxpz2\") pod \"49d51f62-1699-40d6-8714-3e863f78e99c\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") "
Jul 15 21:34:20.963724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.963713    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d51f62-1699-40d6-8714-3e863f78e99c-openshift-service-ca\") pod \"49d51f62-1699-40d6-8714-3e863f78e99c\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") "
Jul 15 21:34:20.963907 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.963788    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") pod \"49d51f62-1699-40d6-8714-3e863f78e99c\" (UID: \"49d51f62-1699-40d6-8714-3e863f78e99c\") "
Jul 15 21:34:20.964098 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.964073    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d51f62-1699-40d6-8714-3e863f78e99c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49d51f62-1699-40d6-8714-3e863f78e99c" (UID: "49d51f62-1699-40d6-8714-3e863f78e99c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:34:20.965818 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.965798    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d51f62-1699-40d6-8714-3e863f78e99c-kube-api-access-mxpz2" (OuterVolumeSpecName: "kube-api-access-mxpz2") pod "49d51f62-1699-40d6-8714-3e863f78e99c" (UID: "49d51f62-1699-40d6-8714-3e863f78e99c"). InnerVolumeSpecName "kube-api-access-mxpz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:34:20.965877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:20.965802    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49d51f62-1699-40d6-8714-3e863f78e99c" (UID: "49d51f62-1699-40d6-8714-3e863f78e99c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:34:21.064606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.064577    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxpz2\" (UniqueName: \"kubernetes.io/projected/49d51f62-1699-40d6-8714-3e863f78e99c-kube-api-access-mxpz2\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:21.064606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.064602    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d51f62-1699-40d6-8714-3e863f78e99c-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:21.064606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.064611    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d51f62-1699-40d6-8714-3e863f78e99c-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:34:21.596463 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.596421    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:21.800244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.800162    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"
Jul 15 21:34:21.800244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.800169    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w" event={"ID":"49d51f62-1699-40d6-8714-3e863f78e99c","Type":"ContainerDied","Data":"1545f06b7bddec22bf51b68ef53046b08fe347da7568ebf732034f25d3ee5b9c"}
Jul 15 21:34:21.800244 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.800233    2555 scope.go:117] "RemoveContainer" containerID="52a74fb4ed94d5da5788e546bf9ea6f72ac6cc21a1681c6f3e5525d8bd9b0ade"
Jul 15 21:34:21.823255 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.823233    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"]
Jul 15 21:34:21.831986 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:21.831950    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2c26b6eb-59d67b9c47-8jl8w"]
Jul 15 21:34:22.218479 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:22.218439    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d51f62-1699-40d6-8714-3e863f78e99c" path="/var/lib/kubelet/pods/49d51f62-1699-40d6-8714-3e863f78e99c/volumes"
Jul 15 21:34:22.219055 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:22.219037    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b24203-7a17-491a-b7f7-0d7ae6434257" path="/var/lib/kubelet/pods/f9b24203-7a17-491a-b7f7-0d7ae6434257/volumes"
Jul 15 21:34:28.330290 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:28.330252    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:28.401179 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:28.401146    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:28.855113 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:28.855084    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:28.951116 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:28.951087    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:29.047799 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:29.047772    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:30.521305 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:30.521273    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:37.174094 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:37.174055    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:37.286077 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:37.286039    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:37.384907 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:37.384877    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:37.482180 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:37.482085    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:34:38.008825 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:34:38.008795    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:04.010153 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:04.010076    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:10.294103 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:10.294070    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:10.355149 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:10.355118    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:10.440739 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:10.440696    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:10.539047 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:10.539020    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:26.417643 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:26.417604    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:33.312484 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:33.312451    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:33.412107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:33.412073    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:33.503422 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:33.503391    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:44.007495 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:44.007446    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:44.848815 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:44.848766    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:50.129079 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:50.129045    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:50.230755 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:50.230725    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:51.448111 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:51.448084    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:51.522824 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:51.522790    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:51.619762 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:51.619734    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:51.700700 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:51.700629    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:51.800111 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:51.800087    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:52.528471 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:52.528433    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:52.628895 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:52.628864    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:52.730921 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:52.730894    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:35:54.388248 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:35:54.388209    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:00.998065 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:00.998033    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:01.061017 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:01.060983    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:01.720519 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:01.720480    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:08.012201 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.012165    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:08.167928 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.167899    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:08.308763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.308681    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:08.621116 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621023    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"]
Jul 15 21:36:08.621475 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621457    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b24203-7a17-491a-b7f7-0d7ae6434257" containerName="istio-proxy"
Jul 15 21:36:08.621606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621478    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b24203-7a17-491a-b7f7-0d7ae6434257" containerName="istio-proxy"
Jul 15 21:36:08.621606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621515    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d51f62-1699-40d6-8714-3e863f78e99c" containerName="maas-api"
Jul 15 21:36:08.621606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621524    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d51f62-1699-40d6-8714-3e863f78e99c" containerName="maas-api"
Jul 15 21:36:08.621606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621546    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="675f1adc-c0bd-469d-8a2f-238938997820" containerName="istio-proxy"
Jul 15 21:36:08.621606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621555    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="675f1adc-c0bd-469d-8a2f-238938997820" containerName="istio-proxy"
Jul 15 21:36:08.621850 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621644    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d51f62-1699-40d6-8714-3e863f78e99c" containerName="maas-api"
Jul 15 21:36:08.621850 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621660    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b24203-7a17-491a-b7f7-0d7ae6434257" containerName="istio-proxy"
Jul 15 21:36:08.621850 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.621673    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="675f1adc-c0bd-469d-8a2f-238938997820" containerName="istio-proxy"
Jul 15 21:36:08.624711 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.624687    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.627874 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.627852    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1c61c321-openshift-default-dockercfg-fvl2t\""
Jul 15 21:36:08.627874 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.627859    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 21:36:08.640124 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.640093    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:08.654543 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.654516    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"]
Jul 15 21:36:08.723387 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723346    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-envoy\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723401    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-data\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723517    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-token\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723573    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-certs\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723608    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plgm8\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-kube-api-access-plgm8\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723663    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-socket\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-credential-socket\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723737    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-podinfo\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.723987 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.723759    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istiod-ca-cert\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825170 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825133    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-certs\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825335 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825206    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plgm8\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-kube-api-access-plgm8\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825335 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825251    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-socket\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825335 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825284    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-credential-socket\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825335 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825322    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-podinfo\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825558 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825345    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istiod-ca-cert\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825618 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825547    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-certs\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825618 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825598    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-envoy\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825730 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825641    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-socket\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825730 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825645    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-data\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825732    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-token\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825826 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825743    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-credential-socket\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.825932 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.825845    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-data\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.826363 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.826338    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istiod-ca-cert\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.827821 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.827791    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-envoy\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.828069 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.828050    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-podinfo\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.835819 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.835798    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plgm8\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-kube-api-access-plgm8\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.835936 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.835817    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-token\") pod \"e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") " pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:08.936120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:08.936086    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:09.020355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.020323    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:09.073720 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.073699    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"]
Jul 15 21:36:09.075353 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:36:09.075324    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fef24e_6eda_4227_a9a2_b3b891b909e8.slice/crio-daf7f06e92da496f550f466d6294a587797dcff322aed50613cbed69850695f6 WatchSource:0}: Error finding container daf7f06e92da496f550f466d6294a587797dcff322aed50613cbed69850695f6: Status 404 returned error can't find the container with id daf7f06e92da496f550f466d6294a587797dcff322aed50613cbed69850695f6
Jul 15 21:36:09.077503 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.077471    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:36:09.077587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.077534    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:36:09.077587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.077562    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:36:09.306454 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.306357    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" event={"ID":"e2fef24e-6eda-4227-a9a2-b3b891b909e8","Type":"ContainerStarted","Data":"a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"}
Jul 15 21:36:09.306454 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.306425    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" event={"ID":"e2fef24e-6eda-4227-a9a2-b3b891b909e8","Type":"ContainerStarted","Data":"daf7f06e92da496f550f466d6294a587797dcff322aed50613cbed69850695f6"}
Jul 15 21:36:09.336945 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.336888    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" podStartSLOduration=1.336864794 podStartE2EDuration="1.336864794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:36:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:36:09.335123345 +0000 UTC m=+1773.667264455" watchObservedRunningTime="2026-07-15 21:36:09.336864794 +0000 UTC m=+1773.669005884"
Jul 15 21:36:09.936351 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:09.936312    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:10.383994 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.383931    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:10.483210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.483175    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:10.600719 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.600682    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"]
Jul 15 21:36:10.604238 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.604215    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.606660 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.606639    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1c61c321-serving-cert\""
Jul 15 21:36:10.606840 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.606821    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:36:10.617725 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.617704    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"]
Jul 15 21:36:10.641792 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.641698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44qzb\" (UniqueName: \"kubernetes.io/projected/a373412c-3c87-46d0-85df-951540158ea0-kube-api-access-44qzb\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.641792 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.641743    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a373412c-3c87-46d0-85df-951540158ea0-openshift-service-ca\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.641792 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.641774    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.646362 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.646336    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:10.742876 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.742840    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44qzb\" (UniqueName: \"kubernetes.io/projected/a373412c-3c87-46d0-85df-951540158ea0-kube-api-access-44qzb\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.743054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.742895    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a373412c-3c87-46d0-85df-951540158ea0-openshift-service-ca\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.743054 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.742938    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.743168 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:36:10.743059    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-serving-cert: secret "maas-api-e2e-mt-1c61c321-serving-cert" not found
Jul 15 21:36:10.743168 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:36:10.743124    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls podName:a373412c-3c87-46d0-85df-951540158ea0 nodeName:}" failed. No retries permitted until 2026-07-15 21:36:11.243100245 +0000 UTC m=+1775.575241320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls") pod "maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" (UID: "a373412c-3c87-46d0-85df-951540158ea0") : secret "maas-api-e2e-mt-1c61c321-serving-cert" not found
Jul 15 21:36:10.743587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.743568    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a373412c-3c87-46d0-85df-951540158ea0-openshift-service-ca\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.752988 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.752930    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44qzb\" (UniqueName: \"kubernetes.io/projected/a373412c-3c87-46d0-85df-951540158ea0-kube-api-access-44qzb\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:10.942014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.941912    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:10.973077 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:10.973041    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:11.158640 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.158607    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:11.247861 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.247777    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:11.250324 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.250292    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls\") pod \"maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:11.257573 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.257550    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:11.321074 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.321041    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:11.325741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.325705    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:11.515359 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.515260    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:11.659229 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:11.659202    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"]
Jul 15 21:36:11.660909 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:36:11.660882    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda373412c_3c87_46d0_85df_951540158ea0.slice/crio-1b0420865aaaca0ec8108d2c01c146ce4b0ffc3185f69175b80f36f1c8c2efaf WatchSource:0}: Error finding container 1b0420865aaaca0ec8108d2c01c146ce4b0ffc3185f69175b80f36f1c8c2efaf: Status 404 returned error can't find the container with id 1b0420865aaaca0ec8108d2c01c146ce4b0ffc3185f69175b80f36f1c8c2efaf
Jul 15 21:36:12.327010 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:12.326942    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" event={"ID":"a373412c-3c87-46d0-85df-951540158ea0","Type":"ContainerStarted","Data":"1b0420865aaaca0ec8108d2c01c146ce4b0ffc3185f69175b80f36f1c8c2efaf"}
Jul 15 21:36:13.333680 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:13.333642    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" event={"ID":"a373412c-3c87-46d0-85df-951540158ea0","Type":"ContainerStarted","Data":"1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31"}
Jul 15 21:36:13.334111 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:13.333716    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:13.353457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:13.353406    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" podStartSLOduration=2.64556139 podStartE2EDuration="3.353389697s" totalImagesPullingTime="707.828307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:36:13.35251535 +0000 UTC m=+1777.684656439" watchObservedRunningTime="2026-07-15 21:36:13.353389697 +0000 UTC m=+1777.685530786"
Jul 15 21:36:15.880677 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:15.880643    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:16.549403 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:16.549371    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:16.649428 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:16.649394    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:16.988015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:16.987973    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:19.345725 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:19.345693    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:24.212612 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:24.212567    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:24.321313 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:24.321275    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:24.996659 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:24.996629    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:25.401117 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:25.401090    2555 generic.go:356] "Generic (PLEG): container finished" podID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerID="a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693" exitCode=0
Jul 15 21:36:25.401561 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:25.401154    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" event={"ID":"e2fef24e-6eda-4227-a9a2-b3b891b909e8","Type":"ContainerDied","Data":"a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"}
Jul 15 21:36:25.401561 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:25.401530    2555 scope.go:117] "RemoveContainer" containerID="a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"
Jul 15 21:36:25.402488 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:25.402460    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:36:25.402590 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:25.402532    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:36:25.402590 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:25.402562    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:36:26.321709 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:26.321673    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:26.407705 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:26.407672    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" event={"ID":"e2fef24e-6eda-4227-a9a2-b3b891b909e8","Type":"ContainerStarted","Data":"939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac"}
Jul 15 21:36:26.938376 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:26.938341    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:26.938800 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:26.938775    2555 patch_prober.go:28] interesting pod/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 21:36:26.939016 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:26.938988    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 21:36:27.937210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:27.936682    2555 patch_prober.go:28] interesting pod/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 21:36:27.937210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:27.936756    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 21:36:28.937080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:28.937042    2555 patch_prober.go:28] interesting pod/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 21:36:28.937285 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:28.937117    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 21:36:29.936711 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:29.936667    2555 patch_prober.go:28] interesting pod/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 15 21:36:29.936910 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:29.936733    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 15 21:36:30.940155 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:30.940127    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:30.940607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:30.940390    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:30.941033 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:30.941017    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:31.016306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.016262    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"]
Jul 15 21:36:31.016763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.016734    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" podUID="a373412c-3c87-46d0-85df-951540158ea0" containerName="maas-api" containerID="cri-o://1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31" gracePeriod=30
Jul 15 21:36:31.070292 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.070261    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:31.265284 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.265259    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:31.336268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.336239    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a373412c-3c87-46d0-85df-951540158ea0-openshift-service-ca\") pod \"a373412c-3c87-46d0-85df-951540158ea0\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") "
Jul 15 21:36:31.336421 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.336301    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44qzb\" (UniqueName: \"kubernetes.io/projected/a373412c-3c87-46d0-85df-951540158ea0-kube-api-access-44qzb\") pod \"a373412c-3c87-46d0-85df-951540158ea0\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") "
Jul 15 21:36:31.336421 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.336370    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls\") pod \"a373412c-3c87-46d0-85df-951540158ea0\" (UID: \"a373412c-3c87-46d0-85df-951540158ea0\") "
Jul 15 21:36:31.336664 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.336637    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a373412c-3c87-46d0-85df-951540158ea0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a373412c-3c87-46d0-85df-951540158ea0" (UID: "a373412c-3c87-46d0-85df-951540158ea0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:36:31.338390 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.338367    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a373412c-3c87-46d0-85df-951540158ea0" (UID: "a373412c-3c87-46d0-85df-951540158ea0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:36:31.338499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.338369    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a373412c-3c87-46d0-85df-951540158ea0-kube-api-access-44qzb" (OuterVolumeSpecName: "kube-api-access-44qzb") pod "a373412c-3c87-46d0-85df-951540158ea0" (UID: "a373412c-3c87-46d0-85df-951540158ea0"). InnerVolumeSpecName "kube-api-access-44qzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:36:31.436571 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.436532    2555 generic.go:356] "Generic (PLEG): container finished" podID="a373412c-3c87-46d0-85df-951540158ea0" containerID="1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31" exitCode=0
Jul 15 21:36:31.436735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.436599    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"
Jul 15 21:36:31.436735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.436613    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" event={"ID":"a373412c-3c87-46d0-85df-951540158ea0","Type":"ContainerDied","Data":"1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31"}
Jul 15 21:36:31.436735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.436687    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg" event={"ID":"a373412c-3c87-46d0-85df-951540158ea0","Type":"ContainerDied","Data":"1b0420865aaaca0ec8108d2c01c146ce4b0ffc3185f69175b80f36f1c8c2efaf"}
Jul 15 21:36:31.436735 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.436711    2555 scope.go:117] "RemoveContainer" containerID="1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31"
Jul 15 21:36:31.437051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.437028    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a373412c-3c87-46d0-85df-951540158ea0-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:31.437131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.437057    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a373412c-3c87-46d0-85df-951540158ea0-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:31.437131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.437069    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44qzb\" (UniqueName: \"kubernetes.io/projected/a373412c-3c87-46d0-85df-951540158ea0-kube-api-access-44qzb\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:31.446201 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.446184    2555 scope.go:117] "RemoveContainer" containerID="1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31"
Jul 15 21:36:31.446438 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:36:31.446420    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31\": container with ID starting with 1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31 not found: ID does not exist" containerID="1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31"
Jul 15 21:36:31.446502 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.446448    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31"} err="failed to get container status \"1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31\": rpc error: code = NotFound desc = could not find container \"1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31\": container with ID starting with 1602cef45d4f201e23bf097794493ec42ebaa5e92dc557146dad4bb33b56ad31 not found: ID does not exist"
Jul 15 21:36:31.459339 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.459316    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"]
Jul 15 21:36:31.466674 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.466654    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1c61c321-69cc7475ff-mdbgg"]
Jul 15 21:36:31.748883 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:31.748850    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:32.217294 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:32.217262    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a373412c-3c87-46d0-85df-951540158ea0" path="/var/lib/kubelet/pods/a373412c-3c87-46d0-85df-951540158ea0/volumes"
Jul 15 21:36:36.213202 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:36.213174    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:36:36.215559 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:36.215534    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:36:38.338344 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:38.338306    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:38.441720 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:38.441687    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:38.651468 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:38.651430    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"]
Jul 15 21:36:38.651684 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:38.651662    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy" containerID="cri-o://939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac" gracePeriod=30
Jul 15 21:36:38.658919 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:38.658895    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:38.927537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:38.927461    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:39.030484 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:39.030421    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:39.132163 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:39.132133    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:39.232511 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:39.232430    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:43.901193 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:43.901173    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:44.046142 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046064    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-data\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046142 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046118    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istiod-ca-cert\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046154    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-credential-socket\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046173    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-certs\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046188    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-podinfo\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046288    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-token\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046336    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-socket\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046369    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-envoy\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046409    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plgm8\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-kube-api-access-plgm8\") pod \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\" (UID: \"e2fef24e-6eda-4227-a9a2-b3b891b909e8\") "
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046476    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046494    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046687    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046706    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046740    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:36:44.046939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046754    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:36:44.047276 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.046993    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:36:44.048504 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.048469    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:36:44.048601 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.048508    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:36:44.048822 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.048801    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:36:44.048865 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.048818    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-kube-api-access-plgm8" (OuterVolumeSpecName: "kube-api-access-plgm8") pod "e2fef24e-6eda-4227-a9a2-b3b891b909e8" (UID: "e2fef24e-6eda-4227-a9a2-b3b891b909e8"). InnerVolumeSpecName "kube-api-access-plgm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:36:44.147397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147379    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plgm8\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-kube-api-access-plgm8\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.147397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147398    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.147540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147407    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.147540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147416    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.147540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147424    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.147540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147431    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.147540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.147441    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2fef24e-6eda-4227-a9a2-b3b891b909e8-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:36:44.505466 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.505430    2555 generic.go:356] "Generic (PLEG): container finished" podID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerID="939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac" exitCode=0
Jul 15 21:36:44.505642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.505487    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"
Jul 15 21:36:44.505642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.505526    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" event={"ID":"e2fef24e-6eda-4227-a9a2-b3b891b909e8","Type":"ContainerDied","Data":"939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac"}
Jul 15 21:36:44.505642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.505616    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w" event={"ID":"e2fef24e-6eda-4227-a9a2-b3b891b909e8","Type":"ContainerDied","Data":"daf7f06e92da496f550f466d6294a587797dcff322aed50613cbed69850695f6"}
Jul 15 21:36:44.505642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.505641    2555 scope.go:117] "RemoveContainer" containerID="939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac"
Jul 15 21:36:44.513917 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.513896    2555 scope.go:117] "RemoveContainer" containerID="a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"
Jul 15 21:36:44.521253 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.521237    2555 scope.go:117] "RemoveContainer" containerID="939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac"
Jul 15 21:36:44.521507 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:36:44.521484    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac\": container with ID starting with 939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac not found: ID does not exist" containerID="939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac"
Jul 15 21:36:44.521595 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.521512    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac"} err="failed to get container status \"939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac\": rpc error: code = NotFound desc = could not find container \"939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac\": container with ID starting with 939498a0ccf9f0c5cff06e35114d12e6b02e48f16cdee2eea085c02fba5802ac not found: ID does not exist"
Jul 15 21:36:44.521595 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.521529    2555 scope.go:117] "RemoveContainer" containerID="a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"
Jul 15 21:36:44.521762 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:36:44.521744    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693\": container with ID starting with a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693 not found: ID does not exist" containerID="a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"
Jul 15 21:36:44.521805 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.521767    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693"} err="failed to get container status \"a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693\": rpc error: code = NotFound desc = could not find container \"a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693\": container with ID starting with a1c3fd2d34e4cc671baff5eb7d9a2a0e6e40cc165ecfedb771b2ee5abcda0693 not found: ID does not exist"
Jul 15 21:36:44.527279 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.527260    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"]
Jul 15 21:36:44.532371 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:44.532352    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1c61c321-openshift-default-6955cbb44c-lk85w"]
Jul 15 21:36:45.788041 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:45.788007    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:45.855922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:45.855894    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:45.936012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:45.935980    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:46.035864 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:46.035834    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:46.164900 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:46.164871    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:46.217992 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:46.217922    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" path="/var/lib/kubelet/pods/e2fef24e-6eda-4227-a9a2-b3b891b909e8/volumes"
Jul 15 21:36:52.285616 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:52.285581    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:36:52.342465 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:36:52.342432    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:00.046966 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:00.046915    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:00.147531 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:00.147496    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:00.241319 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:00.241286    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:00.969578 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:00.969550    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:01.076388 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:01.076353    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:01.172969 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:01.172920    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:02.653125 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:02.653089    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:09.242512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:09.242473    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:09.304694 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:09.304665    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:09.925646 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:09.925609    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:14.300747 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:14.300711    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:16.357209 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:16.357173    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:16.421421 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:16.421388    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:16.498621 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:16.498584    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:16.591801 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:16.591769    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:27.095009 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:27.094948    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:27.451436 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:27.451402    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:48.404899 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:48.404865    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:54.792272 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:54.792229    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:54.894929 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:54.894895    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:55.200754 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.200723    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:55.554574 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.554491    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:55.576866 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.576834    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"]
Jul 15 21:37:55.577365 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577350    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy"
Jul 15 21:37:55.577420 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577369    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy"
Jul 15 21:37:55.577420 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577384    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy"
Jul 15 21:37:55.577420 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577393    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy"
Jul 15 21:37:55.577515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577435    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a373412c-3c87-46d0-85df-951540158ea0" containerName="maas-api"
Jul 15 21:37:55.577515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577446    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a373412c-3c87-46d0-85df-951540158ea0" containerName="maas-api"
Jul 15 21:37:55.577584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577516    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a373412c-3c87-46d0-85df-951540158ea0" containerName="maas-api"
Jul 15 21:37:55.577584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577532    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy"
Jul 15 21:37:55.577584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.577541    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fef24e-6eda-4227-a9a2-b3b891b909e8" containerName="istio-proxy"
Jul 15 21:37:55.579928 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.579912    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.584125 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.584097    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-99ed79-openshift-default-dockercfg-dm2h9\""
Jul 15 21:37:55.584318 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.584303    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 21:37:55.610511 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.610486    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"]
Jul 15 21:37:55.662868 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.662834    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-socket\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.662868 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.662873    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-credential-socket\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.662922    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-data\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.662986    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-token\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.663007    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cbbce-220f-4504-8af0-265ad8319a7d-istiod-ca-cert\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.663035    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-podinfo\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.663072    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-certs\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663137 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.663137    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-envoy\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.663411 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.663171    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mtpm\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-kube-api-access-7mtpm\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764240 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764204    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mtpm\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-kube-api-access-7mtpm\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764256    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-socket\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764273    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-credential-socket\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764307    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-data\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764336    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-token\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764352    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cbbce-220f-4504-8af0-265ad8319a7d-istiod-ca-cert\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764438 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764370    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-podinfo\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764648    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-certs\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764696    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-socket\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764720    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-credential-socket\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.764768 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764723    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-envoy\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.765015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764844    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-data\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.765015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.764982    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-certs\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.765192 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.765164    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cbbce-220f-4504-8af0-265ad8319a7d-istiod-ca-cert\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.767042 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.767019    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-envoy\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.767164 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.767145    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-podinfo\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.774475 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.774454    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-token\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.774676 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.774655    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mtpm\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-kube-api-access-7mtpm\") pod \"e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") " pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:55.809751 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.809674    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:55.890015 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:55.889971    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:56.027160 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.027133    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"]
Jul 15 21:37:56.029567 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:37:56.029535    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98cbbce_220f_4504_8af0_265ad8319a7d.slice/crio-e0a3e8a6be0b253f36dce498929d5412e12a25566f4b8a691d3f2f7153db92d5 WatchSource:0}: Error finding container e0a3e8a6be0b253f36dce498929d5412e12a25566f4b8a691d3f2f7153db92d5: Status 404 returned error can't find the container with id e0a3e8a6be0b253f36dce498929d5412e12a25566f4b8a691d3f2f7153db92d5
Jul 15 21:37:56.031636 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.031606    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:37:56.031719 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.031670    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:37:56.031719 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.031701    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:37:56.872651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.872613    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" event={"ID":"f98cbbce-220f-4504-8af0-265ad8319a7d","Type":"ContainerStarted","Data":"41b75b851fbcc47e1e61b3053ab3e0a48d81f5827f82c6d15d2cd7c44400e255"}
Jul 15 21:37:56.873115 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.872695    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" event={"ID":"f98cbbce-220f-4504-8af0-265ad8319a7d","Type":"ContainerStarted","Data":"e0a3e8a6be0b253f36dce498929d5412e12a25566f4b8a691d3f2f7153db92d5"}
Jul 15 21:37:56.890228 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.890202    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:56.896414 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:56.896370    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podStartSLOduration=1.8963554459999998 podStartE2EDuration="1.896355446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:56.893643376 +0000 UTC m=+1881.225784460" watchObservedRunningTime="2026-07-15 21:37:56.896355446 +0000 UTC m=+1881.228496535"
Jul 15 21:37:57.808758 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.808637    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:57.894785 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.894749    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"]
Jul 15 21:37:57.896556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.896526    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:57.897470 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.897448    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:57.897575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.897491    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:57.899945 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.899923    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-99ed79-serving-cert\""
Jul 15 21:37:57.900076 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.900058    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:37:57.908623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.908599    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"]
Jul 15 21:37:57.963665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.963627    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:57.985887 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.985855    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ecf533-174d-430b-96bb-2228879a871a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:57.986070 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.985921    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrlg\" (UniqueName: \"kubernetes.io/projected/16ecf533-174d-430b-96bb-2228879a871a-kube-api-access-qnrlg\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:57.986136 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:57.986102    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.087620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.087528    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ecf533-174d-430b-96bb-2228879a871a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.087620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.087580    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrlg\" (UniqueName: \"kubernetes.io/projected/16ecf533-174d-430b-96bb-2228879a871a-kube-api-access-qnrlg\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.087620 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.087619    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.087909 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:37:58.087753    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-serving-cert: secret "maas-api-e2e-inf-a-99ed79-serving-cert" not found
Jul 15 21:37:58.087909 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:37:58.087814    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls podName:16ecf533-174d-430b-96bb-2228879a871a nodeName:}" failed. No retries permitted until 2026-07-15 21:37:58.587797604 +0000 UTC m=+1882.919938675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls") pod "maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" (UID: "16ecf533-174d-430b-96bb-2228879a871a") : secret "maas-api-e2e-inf-a-99ed79-serving-cert" not found
Jul 15 21:37:58.088239 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.088219    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ecf533-174d-430b-96bb-2228879a871a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.097656 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.097625    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrlg\" (UniqueName: \"kubernetes.io/projected/16ecf533-174d-430b-96bb-2228879a871a-kube-api-access-qnrlg\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.353598 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.353509    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:58.509608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.509575    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:58.592662 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.592619    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.595108 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.595087    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls\") pod \"maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.650274 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.650240    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:37:58.810090 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.810051    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:58.883649 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.883624    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:58.884773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:58.884755    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:37:59.153291 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:59.153267    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"]
Jul 15 21:37:59.154691 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:37:59.154662    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ecf533_174d_430b_96bb_2228879a871a.slice/crio-d215ddfcab321c973c29d706279f0329b071d16d5cbe2be71e3a8afa97b191a4 WatchSource:0}: Error finding container d215ddfcab321c973c29d706279f0329b071d16d5cbe2be71e3a8afa97b191a4: Status 404 returned error can't find the container with id d215ddfcab321c973c29d706279f0329b071d16d5cbe2be71e3a8afa97b191a4
Jul 15 21:37:59.889389 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:59.889294    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" event={"ID":"16ecf533-174d-430b-96bb-2228879a871a","Type":"ContainerStarted","Data":"5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962"}
Jul 15 21:37:59.889560 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:59.889410    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:37:59.889560 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:59.889452    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" event={"ID":"16ecf533-174d-430b-96bb-2228879a871a","Type":"ContainerStarted","Data":"d215ddfcab321c973c29d706279f0329b071d16d5cbe2be71e3a8afa97b191a4"}
Jul 15 21:37:59.909266 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:37:59.909204    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" podStartSLOduration=2.43917536 podStartE2EDuration="2.909189409s" totalImagesPullingTime="470.014049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:59.907865026 +0000 UTC m=+1884.240006130" watchObservedRunningTime="2026-07-15 21:37:59.909189409 +0000 UTC m=+1884.241330498"
Jul 15 21:38:03.364110 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.364073    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:03.691628 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.691584    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:03.971976 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.971874    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"]
Jul 15 21:38:03.974793 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.974767    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:03.977377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.977349    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-518fcf-openshift-default-dockercfg-hgk57\""
Jul 15 21:38:03.987021 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.986996    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"]
Jul 15 21:38:03.987461 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:03.987431    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:04.048501 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048463    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd154804-3eed-4048-b453-a23a99c4e18b-istio-podinfo\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.048501 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048502    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dntk\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-kube-api-access-6dntk\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.048789 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048523    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-socket\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.048789 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048634    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-certs\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.048789 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048671    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd154804-3eed-4048-b453-a23a99c4e18b-istiod-ca-cert\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.048789 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048714    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-credential-socket\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.048789 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048778    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-istio-token\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.049134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048816    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-envoy\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.049134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.048852    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-data\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.145982 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.145918    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:04.150109 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150078    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd154804-3eed-4048-b453-a23a99c4e18b-istio-podinfo\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150261 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150114    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dntk\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-kube-api-access-6dntk\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150261 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150144    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-socket\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150261 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150172    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-certs\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150261 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150202    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd154804-3eed-4048-b453-a23a99c4e18b-istiod-ca-cert\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150261 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150243    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-credential-socket\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150307    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-istio-token\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150357    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-envoy\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150517 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150422    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-data\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150679 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150651    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-socket\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.150872 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150847    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-certs\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.151010 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150879    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-credential-socket\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.151010 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.150922    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-data\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.151133 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.151115    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd154804-3eed-4048-b453-a23a99c4e18b-istiod-ca-cert\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.152619 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.152593    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-envoy\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.152844 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.152824    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd154804-3eed-4048-b453-a23a99c4e18b-istio-podinfo\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.159832 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.159808    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dntk\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-kube-api-access-6dntk\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.160150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.160130    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-istio-token\") pod \"e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") " pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.288406 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.288310    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:04.426370 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.426344    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"]
Jul 15 21:38:04.429138 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:38:04.429103    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd154804_3eed_4048_b453_a23a99c4e18b.slice/crio-f0da51187ca67d30bb615e6162407ad07182db391834b678ff939fb93927fd32 WatchSource:0}: Error finding container f0da51187ca67d30bb615e6162407ad07182db391834b678ff939fb93927fd32: Status 404 returned error can't find the container with id f0da51187ca67d30bb615e6162407ad07182db391834b678ff939fb93927fd32
Jul 15 21:38:04.432056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.432025    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:04.432151 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.432097    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:04.432151 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.432130    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:04.914423 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.914391    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" event={"ID":"dd154804-3eed-4048-b453-a23a99c4e18b","Type":"ContainerStarted","Data":"35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"}
Jul 15 21:38:04.914608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.914443    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" event={"ID":"dd154804-3eed-4048-b453-a23a99c4e18b","Type":"ContainerStarted","Data":"f0da51187ca67d30bb615e6162407ad07182db391834b678ff939fb93927fd32"}
Jul 15 21:38:04.940066 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:04.940013    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podStartSLOduration=1.9399910999999999 podStartE2EDuration="1.9399911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:38:04.937623911 +0000 UTC m=+1889.269765000" watchObservedRunningTime="2026-07-15 21:38:04.9399911 +0000 UTC m=+1889.272132190"
Jul 15 21:38:05.288829 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:05.288730    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:05.293711 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:05.293684    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:05.898888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:05.898853    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:38:05.920607 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:05.920569    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:05.921926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:05.921888    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:05.971401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:05.971373    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:06.060069 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.060039    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:06.157810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.157725    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:06.222617 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.222585    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"]
Jul 15 21:38:06.226638 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.226611    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.229575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.229546    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-518fcf-serving-cert\""
Jul 15 21:38:06.238259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.237766    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"]
Jul 15 21:38:06.268101 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.268055    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:06.272231 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.272206    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.272321 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.272254    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3ad3500-c5e4-4945-8087-324ee6c06618-openshift-service-ca\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.272407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.272388    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxw77\" (UniqueName: \"kubernetes.io/projected/d3ad3500-c5e4-4945-8087-324ee6c06618-kube-api-access-cxw77\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.373783 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.373746    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxw77\" (UniqueName: \"kubernetes.io/projected/d3ad3500-c5e4-4945-8087-324ee6c06618-kube-api-access-cxw77\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.374018 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.373804    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.374018 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.373878    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3ad3500-c5e4-4945-8087-324ee6c06618-openshift-service-ca\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.374018 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:38:06.373900    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-serving-cert: secret "maas-api-e2e-inf-b-518fcf-serving-cert" not found
Jul 15 21:38:06.374018 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:38:06.373994    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls podName:d3ad3500-c5e4-4945-8087-324ee6c06618 nodeName:}" failed. No retries permitted until 2026-07-15 21:38:06.873941654 +0000 UTC m=+1891.206082722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls") pod "maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" (UID: "d3ad3500-c5e4-4945-8087-324ee6c06618") : secret "maas-api-e2e-inf-b-518fcf-serving-cert" not found
Jul 15 21:38:06.374649 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.374625    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3ad3500-c5e4-4945-8087-324ee6c06618-openshift-service-ca\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.386712 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.386689    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxw77\" (UniqueName: \"kubernetes.io/projected/d3ad3500-c5e4-4945-8087-324ee6c06618-kube-api-access-cxw77\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.708288 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.708251    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:06.880418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.880384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.883030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.882998    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls\") pod \"maas-api-e2e-inf-b-518fcf-6946856897-b5hwl\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:06.900716 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.900689    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:06.994381 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:06.994281    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:07.142129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:07.142095    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:07.285111 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:07.285085    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"]
Jul 15 21:38:07.287015 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:38:07.286991    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ad3500_c5e4_4945_8087_324ee6c06618.slice/crio-d619ce4926723ef7b2239abeb3546d7023b0a40b98de286c22a0363656588b8b WatchSource:0}: Error finding container d619ce4926723ef7b2239abeb3546d7023b0a40b98de286c22a0363656588b8b: Status 404 returned error can't find the container with id d619ce4926723ef7b2239abeb3546d7023b0a40b98de286c22a0363656588b8b
Jul 15 21:38:07.944514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:07.944476    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" event={"ID":"d3ad3500-c5e4-4945-8087-324ee6c06618","Type":"ContainerStarted","Data":"8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c"}
Jul 15 21:38:07.945347 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:07.945243    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" event={"ID":"d3ad3500-c5e4-4945-8087-324ee6c06618","Type":"ContainerStarted","Data":"d619ce4926723ef7b2239abeb3546d7023b0a40b98de286c22a0363656588b8b"}
Jul 15 21:38:07.945347 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:07.945304    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:07.967761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:07.967713    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" podStartSLOduration=1.642040575 podStartE2EDuration="1.967696978s" totalImagesPullingTime="325.656403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:38:07.965921 +0000 UTC m=+1892.298062092" watchObservedRunningTime="2026-07-15 21:38:07.967696978 +0000 UTC m=+1892.299838067"
Jul 15 21:38:11.440123 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:11.439946    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:13.955060 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:13.955031    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:38:14.414373 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.413931    2555 patch_prober.go:28] interesting pod/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:35626->10.134.0.50:15021: read: connection reset by peer" start-of-body=
Jul 15 21:38:14.414373 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.414034    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:35626->10.134.0.50:15021: read: connection reset by peer"
Jul 15 21:38:14.483680 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.483647    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:14.981181 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.981153    2555 generic.go:356] "Generic (PLEG): container finished" podID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerID="41b75b851fbcc47e1e61b3053ab3e0a48d81f5827f82c6d15d2cd7c44400e255" exitCode=0
Jul 15 21:38:14.981586 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.981204    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" event={"ID":"f98cbbce-220f-4504-8af0-265ad8319a7d","Type":"ContainerDied","Data":"41b75b851fbcc47e1e61b3053ab3e0a48d81f5827f82c6d15d2cd7c44400e255"}
Jul 15 21:38:14.981586 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.981547    2555 scope.go:117] "RemoveContainer" containerID="41b75b851fbcc47e1e61b3053ab3e0a48d81f5827f82c6d15d2cd7c44400e255"
Jul 15 21:38:14.982520 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.982494    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:14.982589 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.982563    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:14.982630 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:14.982592    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:15.987862 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:15.987830    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" event={"ID":"f98cbbce-220f-4504-8af0-265ad8319a7d","Type":"ContainerStarted","Data":"551d20dccf9c1b1ad9ceb87da1cd52483f6e4ec790bb6d6072758696dac39ed4"}
Jul 15 21:38:16.890593 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:16.890562    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:38:16.890772 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:16.890736    2555 patch_prober.go:28] interesting pod/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 21:38:16.890821 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:16.890780    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 21:38:17.890626 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:17.890585    2555 patch_prober.go:28] interesting pod/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 21:38:17.891126 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:17.890652    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 21:38:18.891243 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:18.891209    2555 patch_prober.go:28] interesting pod/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 15 21:38:18.891650 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:18.891262    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 15 21:38:19.404056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:19.404021    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:19.505152 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:19.505121    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:19.955715 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:19.955678    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:38:19.956320 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:19.956086    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:38:19.957293 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:19.957273    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:38:21.962117 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:21.962080    2555 patch_prober.go:28] interesting pod/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 15 21:38:21.962515 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:21.962157    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 21:38:24.034838 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:24.034800    2555 generic.go:356] "Generic (PLEG): container finished" podID="dd154804-3eed-4048-b453-a23a99c4e18b" containerID="35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1" exitCode=0
Jul 15 21:38:24.035381 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:24.034943    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" event={"ID":"dd154804-3eed-4048-b453-a23a99c4e18b","Type":"ContainerDied","Data":"35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"}
Jul 15 21:38:24.035549 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:24.035527    2555 scope.go:117] "RemoveContainer" containerID="35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"
Jul 15 21:38:24.037014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:24.036973    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:24.037535 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:24.037152    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:24.037535 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:24.037276    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:38:25.045840 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:25.045797    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" event={"ID":"dd154804-3eed-4048-b453-a23a99c4e18b","Type":"ContainerStarted","Data":"389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3"}
Jul 15 21:38:25.288835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:25.288799    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:25.289034 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:25.288998    2555 patch_prober.go:28] interesting pod/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 15 21:38:25.289093 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:25.289042    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 15 21:38:26.289566 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:26.289530    2555 patch_prober.go:28] interesting pod/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 15 21:38:26.289982 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:26.289614    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 15 21:38:27.289596 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:27.289560    2555 patch_prober.go:28] interesting pod/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 15 21:38:27.290010 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:27.289614    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 15 21:38:28.288939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:28.288906    2555 patch_prober.go:28] interesting pod/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 15 21:38:28.289131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:28.288974    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 15 21:38:28.884645 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:28.884608    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:29.292051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:29.291981    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:29.292357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:29.292343    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:29.292939 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:29.292917    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:38:29.374584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:29.374552    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:38:48.631514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:38:48.631480    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:39:00.577290 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:39:00.577255    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:39:25.381968 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:39:25.381916    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:39:26.323089 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:39:26.323050    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:39:26.820361 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:39:26.820321    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:39:31.971721 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:39:31.971633    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:08.937457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:08.937426    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"]
Jul 15 21:40:08.938028 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:08.937643    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" podUID="16ecf533-174d-430b-96bb-2228879a871a" containerName="maas-api" containerID="cri-o://5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962" gracePeriod=30
Jul 15 21:40:08.953868 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:08.953843    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:09.186182 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.186159    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:40:09.273126 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.273052    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ecf533-174d-430b-96bb-2228879a871a-openshift-service-ca\") pod \"16ecf533-174d-430b-96bb-2228879a871a\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") "
Jul 15 21:40:09.273277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.273126    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnrlg\" (UniqueName: \"kubernetes.io/projected/16ecf533-174d-430b-96bb-2228879a871a-kube-api-access-qnrlg\") pod \"16ecf533-174d-430b-96bb-2228879a871a\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") "
Jul 15 21:40:09.273277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.273158    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls\") pod \"16ecf533-174d-430b-96bb-2228879a871a\" (UID: \"16ecf533-174d-430b-96bb-2228879a871a\") "
Jul 15 21:40:09.273457 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.273439    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ecf533-174d-430b-96bb-2228879a871a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16ecf533-174d-430b-96bb-2228879a871a" (UID: "16ecf533-174d-430b-96bb-2228879a871a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:40:09.275173 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.275139    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ecf533-174d-430b-96bb-2228879a871a-kube-api-access-qnrlg" (OuterVolumeSpecName: "kube-api-access-qnrlg") pod "16ecf533-174d-430b-96bb-2228879a871a" (UID: "16ecf533-174d-430b-96bb-2228879a871a"). InnerVolumeSpecName "kube-api-access-qnrlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:40:09.275294 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.275207    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16ecf533-174d-430b-96bb-2228879a871a" (UID: "16ecf533-174d-430b-96bb-2228879a871a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:40:09.373971 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.373928    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ecf533-174d-430b-96bb-2228879a871a-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:09.373971 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.373975    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnrlg\" (UniqueName: \"kubernetes.io/projected/16ecf533-174d-430b-96bb-2228879a871a-kube-api-access-qnrlg\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:09.374214 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.373990    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ecf533-174d-430b-96bb-2228879a871a-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:09.572236 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.572154    2555 generic.go:356] "Generic (PLEG): container finished" podID="16ecf533-174d-430b-96bb-2228879a871a" containerID="5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962" exitCode=0
Jul 15 21:40:09.572236 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.572201    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"
Jul 15 21:40:09.572419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.572241    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" event={"ID":"16ecf533-174d-430b-96bb-2228879a871a","Type":"ContainerDied","Data":"5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962"}
Jul 15 21:40:09.572419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.572292    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn" event={"ID":"16ecf533-174d-430b-96bb-2228879a871a","Type":"ContainerDied","Data":"d215ddfcab321c973c29d706279f0329b071d16d5cbe2be71e3a8afa97b191a4"}
Jul 15 21:40:09.572419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.572317    2555 scope.go:117] "RemoveContainer" containerID="5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962"
Jul 15 21:40:09.582053 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.582030    2555 scope.go:117] "RemoveContainer" containerID="5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962"
Jul 15 21:40:09.582295 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:40:09.582279    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962\": container with ID starting with 5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962 not found: ID does not exist" containerID="5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962"
Jul 15 21:40:09.582364 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.582306    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962"} err="failed to get container status \"5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962\": rpc error: code = NotFound desc = could not find container \"5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962\": container with ID starting with 5d747518ee3b3c51c58716fb03d7f8cfee842bffbbc6b6b9a68f9236d46d5962 not found: ID does not exist"
Jul 15 21:40:09.597055 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.597030    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"]
Jul 15 21:40:09.602938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.602918    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-99ed79-cf94b5f48-cc8vn"]
Jul 15 21:40:09.817439 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:09.817404    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:10.217660 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:10.217629    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ecf533-174d-430b-96bb-2228879a871a" path="/var/lib/kubelet/pods/16ecf533-174d-430b-96bb-2228879a871a/volumes"
Jul 15 21:40:16.178851 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:16.178822    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:16.239248 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:16.239214    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:16.453335 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:16.453240    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"]
Jul 15 21:40:16.453501 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:16.453440    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy" containerID="cri-o://551d20dccf9c1b1ad9ceb87da1cd52483f6e4ec790bb6d6072758696dac39ed4" gracePeriod=30
Jul 15 21:40:16.463585 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:16.463559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:16.702618 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:16.702581    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:17.193672 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:17.193643    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:19.958449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:19.958400    2555 patch_prober.go:28] interesting pod/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 21:40:19.958830 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:19.958469    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 21:40:21.638455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.638422    2555 generic.go:356] "Generic (PLEG): container finished" podID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerID="551d20dccf9c1b1ad9ceb87da1cd52483f6e4ec790bb6d6072758696dac39ed4" exitCode=0
Jul 15 21:40:21.638810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.638485    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" event={"ID":"f98cbbce-220f-4504-8af0-265ad8319a7d","Type":"ContainerDied","Data":"551d20dccf9c1b1ad9ceb87da1cd52483f6e4ec790bb6d6072758696dac39ed4"}
Jul 15 21:40:21.638810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.638549    2555 scope.go:117] "RemoveContainer" containerID="41b75b851fbcc47e1e61b3053ab3e0a48d81f5827f82c6d15d2cd7c44400e255"
Jul 15 21:40:21.710329 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.710306    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:40:21.885159 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885133    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-token\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885202    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-socket\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885223    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-envoy\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885247    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-podinfo\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885268    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-certs\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885302    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mtpm\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-kube-api-access-7mtpm\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885332    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-data\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885398    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cbbce-220f-4504-8af0-265ad8319a7d-istiod-ca-cert\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885454    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-credential-socket\") pod \"f98cbbce-220f-4504-8af0-265ad8319a7d\" (UID: \"f98cbbce-220f-4504-8af0-265ad8319a7d\") "
Jul 15 21:40:21.885685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885593    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:21.885831 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885787    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.885831 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885600    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:21.885947 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.885844    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:21.886206 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.886172    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f98cbbce-220f-4504-8af0-265ad8319a7d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:40:21.886349 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.886328    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:21.887553 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.887524    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:40:21.887684 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.887553    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:21.887817 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.887797    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:40:21.887892 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.887869    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-kube-api-access-7mtpm" (OuterVolumeSpecName: "kube-api-access-7mtpm") pod "f98cbbce-220f-4504-8af0-265ad8319a7d" (UID: "f98cbbce-220f-4504-8af0-265ad8319a7d"). InnerVolumeSpecName "kube-api-access-7mtpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:40:21.987065 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987032    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f98cbbce-220f-4504-8af0-265ad8319a7d-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987065 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987059    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987065 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987069    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987079    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987086    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987095    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987103    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mtpm\" (UniqueName: \"kubernetes.io/projected/f98cbbce-220f-4504-8af0-265ad8319a7d-kube-api-access-7mtpm\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:21.987297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:21.987112    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f98cbbce-220f-4504-8af0-265ad8319a7d-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:22.645043 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:22.645011    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd" event={"ID":"f98cbbce-220f-4504-8af0-265ad8319a7d","Type":"ContainerDied","Data":"e0a3e8a6be0b253f36dce498929d5412e12a25566f4b8a691d3f2f7153db92d5"}
Jul 15 21:40:22.645471 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:22.645049    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"
Jul 15 21:40:22.645471 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:22.645069    2555 scope.go:117] "RemoveContainer" containerID="551d20dccf9c1b1ad9ceb87da1cd52483f6e4ec790bb6d6072758696dac39ed4"
Jul 15 21:40:22.665425 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:22.665402    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"]
Jul 15 21:40:22.670941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:22.670919    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-99ed79-openshift-default-7459dd6f8c-4blqd"]
Jul 15 21:40:23.619948 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.619908    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"]
Jul 15 21:40:23.620236 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.620185    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" podUID="d3ad3500-c5e4-4945-8087-324ee6c06618" containerName="maas-api" containerID="cri-o://8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c" gracePeriod=30
Jul 15 21:40:23.637530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.637481    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:23.842250 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.842217    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:23.871581 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.871522    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:40:23.900745 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.900715    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls\") pod \"d3ad3500-c5e4-4945-8087-324ee6c06618\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") "
Jul 15 21:40:23.900919 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.900758    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3ad3500-c5e4-4945-8087-324ee6c06618-openshift-service-ca\") pod \"d3ad3500-c5e4-4945-8087-324ee6c06618\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") "
Jul 15 21:40:23.900919 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.900797    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxw77\" (UniqueName: \"kubernetes.io/projected/d3ad3500-c5e4-4945-8087-324ee6c06618-kube-api-access-cxw77\") pod \"d3ad3500-c5e4-4945-8087-324ee6c06618\" (UID: \"d3ad3500-c5e4-4945-8087-324ee6c06618\") "
Jul 15 21:40:23.901219 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.901193    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3ad3500-c5e4-4945-8087-324ee6c06618-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3ad3500-c5e4-4945-8087-324ee6c06618" (UID: "d3ad3500-c5e4-4945-8087-324ee6c06618"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:40:23.903450 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.903420    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3ad3500-c5e4-4945-8087-324ee6c06618" (UID: "d3ad3500-c5e4-4945-8087-324ee6c06618"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:40:23.903583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:23.903446    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ad3500-c5e4-4945-8087-324ee6c06618-kube-api-access-cxw77" (OuterVolumeSpecName: "kube-api-access-cxw77") pod "d3ad3500-c5e4-4945-8087-324ee6c06618" (UID: "d3ad3500-c5e4-4945-8087-324ee6c06618"). InnerVolumeSpecName "kube-api-access-cxw77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:40:24.001514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.001482    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxw77\" (UniqueName: \"kubernetes.io/projected/d3ad3500-c5e4-4945-8087-324ee6c06618-kube-api-access-cxw77\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:24.001514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.001506    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3ad3500-c5e4-4945-8087-324ee6c06618-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:24.001514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.001515    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3ad3500-c5e4-4945-8087-324ee6c06618-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:24.218440 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.218407    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" path="/var/lib/kubelet/pods/f98cbbce-220f-4504-8af0-265ad8319a7d/volumes"
Jul 15 21:40:24.657202 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.657167    2555 generic.go:356] "Generic (PLEG): container finished" podID="d3ad3500-c5e4-4945-8087-324ee6c06618" containerID="8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c" exitCode=0
Jul 15 21:40:24.657398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.657202    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" event={"ID":"d3ad3500-c5e4-4945-8087-324ee6c06618","Type":"ContainerDied","Data":"8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c"}
Jul 15 21:40:24.657398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.657238    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"
Jul 15 21:40:24.657398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.657271    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl" event={"ID":"d3ad3500-c5e4-4945-8087-324ee6c06618","Type":"ContainerDied","Data":"d619ce4926723ef7b2239abeb3546d7023b0a40b98de286c22a0363656588b8b"}
Jul 15 21:40:24.657398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.657295    2555 scope.go:117] "RemoveContainer" containerID="8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c"
Jul 15 21:40:24.666041 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.665995    2555 scope.go:117] "RemoveContainer" containerID="8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c"
Jul 15 21:40:24.666336 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:40:24.666312    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c\": container with ID starting with 8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c not found: ID does not exist" containerID="8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c"
Jul 15 21:40:24.666402 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.666348    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c"} err="failed to get container status \"8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c\": rpc error: code = NotFound desc = could not find container \"8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c\": container with ID starting with 8b50eb1acf81adabc8250f07c3faa98f1b771f0d307295b436c4f55aedac1d4c not found: ID does not exist"
Jul 15 21:40:24.696441 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.696412    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"]
Jul 15 21:40:24.703927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:24.703904    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-518fcf-6946856897-b5hwl"]
Jul 15 21:40:26.219225 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:26.219189    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ad3500-c5e4-4945-8087-324ee6c06618" path="/var/lib/kubelet/pods/d3ad3500-c5e4-4945-8087-324ee6c06618/volumes"
Jul 15 21:40:30.417669 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:30.417631    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:30.686277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:30.686173    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"]
Jul 15 21:40:30.686452 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:30.686426    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy" containerID="cri-o://389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3" gracePeriod=30
Jul 15 21:40:31.589206 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589166    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"]
Jul 15 21:40:31.589741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589720    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy"
Jul 15 21:40:31.589827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589743    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy"
Jul 15 21:40:31.589827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589759    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ecf533-174d-430b-96bb-2228879a871a" containerName="maas-api"
Jul 15 21:40:31.589827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589767    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ecf533-174d-430b-96bb-2228879a871a" containerName="maas-api"
Jul 15 21:40:31.589827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589791    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy"
Jul 15 21:40:31.589827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589801    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy"
Jul 15 21:40:31.589827 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589821    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ad3500-c5e4-4945-8087-324ee6c06618" containerName="maas-api"
Jul 15 21:40:31.590138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589829    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ad3500-c5e4-4945-8087-324ee6c06618" containerName="maas-api"
Jul 15 21:40:31.590138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589920    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy"
Jul 15 21:40:31.590138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589932    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ad3500-c5e4-4945-8087-324ee6c06618" containerName="maas-api"
Jul 15 21:40:31.590138 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.589944    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ecf533-174d-430b-96bb-2228879a871a" containerName="maas-api"
Jul 15 21:40:31.592729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.592705    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.595120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.595097    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-47a933-openshift-default-dockercfg-nnsfc\""
Jul 15 21:40:31.604736 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.604712    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"]
Jul 15 21:40:31.665787 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665755    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-certs\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665809    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-data\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665828    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-istio-token\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665908    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-socket\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665935    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2500a58d-b58c-498e-968b-9fd6e3618678-istio-podinfo\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665973    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2500a58d-b58c-498e-968b-9fd6e3618678-istiod-ca-cert\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.665995    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsz6g\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-kube-api-access-lsz6g\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.666042    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-credential-socket\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.666268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.666070    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-envoy\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.699601 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.699569    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:31.766972 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.766922    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-socket\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.766972 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.766977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2500a58d-b58c-498e-968b-9fd6e3618678-istio-podinfo\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767235 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767210    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2500a58d-b58c-498e-968b-9fd6e3618678-istiod-ca-cert\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767249    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsz6g\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-kube-api-access-lsz6g\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767278    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-credential-socket\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767304    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-envoy\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767315    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-socket\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767368    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-certs\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767445    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-data\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767470    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-istio-token\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767536    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-credential-socket\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767769 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767724    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-certs\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767795    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-data\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.767972 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.767921    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2500a58d-b58c-498e-968b-9fd6e3618678-istiod-ca-cert\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.769514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.769488    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-envoy\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.769598 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.769576    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2500a58d-b58c-498e-968b-9fd6e3618678-istio-podinfo\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.782890 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.782864    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-istio-token\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.783014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.782970    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsz6g\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-kube-api-access-lsz6g\") pod \"e2e-api-a-47a933-openshift-default-56fd85d589-267vh\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") " pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:31.857097 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.857015    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:31.902080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:31.902049    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:32.044342 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.044314    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"]
Jul 15 21:40:32.046214 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:40:32.046191    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2500a58d_b58c_498e_968b_9fd6e3618678.slice/crio-38ccfb075c63e0088f7282791c6de3d15dd596ec864059da0210623f8940a22f WatchSource:0}: Error finding container 38ccfb075c63e0088f7282791c6de3d15dd596ec864059da0210623f8940a22f: Status 404 returned error can't find the container with id 38ccfb075c63e0088f7282791c6de3d15dd596ec864059da0210623f8940a22f
Jul 15 21:40:32.048250 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.048220    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:32.048319 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.048282    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:32.048319 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.048308    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:32.698917 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.698883    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" event={"ID":"2500a58d-b58c-498e-968b-9fd6e3618678","Type":"ContainerStarted","Data":"f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"}
Jul 15 21:40:32.699404 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.698935    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" event={"ID":"2500a58d-b58c-498e-968b-9fd6e3618678","Type":"ContainerStarted","Data":"38ccfb075c63e0088f7282791c6de3d15dd596ec864059da0210623f8940a22f"}
Jul 15 21:40:32.739445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.739390    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" podStartSLOduration=1.739374749 podStartE2EDuration="1.739374749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:32.737610114 +0000 UTC m=+2037.069751205" watchObservedRunningTime="2026-07-15 21:40:32.739374749 +0000 UTC m=+2037.071515838"
Jul 15 21:40:32.902559 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:32.902531    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:33.651252 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.651220    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:33.749271 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.749235    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"]
Jul 15 21:40:33.750426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.750400    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f98cbbce-220f-4504-8af0-265ad8319a7d" containerName="istio-proxy"
Jul 15 21:40:33.752513 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.752494    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.755181 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.755160    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-47a933-serving-cert\""
Jul 15 21:40:33.755295 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.755245    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:40:33.763112 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.763089    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"]
Jul 15 21:40:33.771507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.771485    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:33.785587 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.785558    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ef0e42-ca27-4edc-96cc-487a9da9446d-maas-api-tls\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.785690 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.785618    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-756h4\" (UniqueName: \"kubernetes.io/projected/40ef0e42-ca27-4edc-96cc-487a9da9446d-kube-api-access-756h4\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.785690 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.785644    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ef0e42-ca27-4edc-96cc-487a9da9446d-openshift-service-ca\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.833999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.833948    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:33.886493 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.886459    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ef0e42-ca27-4edc-96cc-487a9da9446d-maas-api-tls\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.886687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.886524    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-756h4\" (UniqueName: \"kubernetes.io/projected/40ef0e42-ca27-4edc-96cc-487a9da9446d-kube-api-access-756h4\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.886687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.886565    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ef0e42-ca27-4edc-96cc-487a9da9446d-openshift-service-ca\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.887321 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.887291    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ef0e42-ca27-4edc-96cc-487a9da9446d-openshift-service-ca\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.889321 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.889298    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ef0e42-ca27-4edc-96cc-487a9da9446d-maas-api-tls\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.897531 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.897493    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-756h4\" (UniqueName: \"kubernetes.io/projected/40ef0e42-ca27-4edc-96cc-487a9da9446d-kube-api-access-756h4\") pod \"maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:33.917197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:33.917115    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:34.064482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.064442    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:34.202326 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.201088    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"]
Jul 15 21:40:34.203860 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:40:34.203817    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ef0e42_ca27_4edc_96cc_487a9da9446d.slice/crio-f3c298ef0050cf22827f7920cc4d01a7b993f79aeb1699006930dbe36e80c572 WatchSource:0}: Error finding container f3c298ef0050cf22827f7920cc4d01a7b993f79aeb1699006930dbe36e80c572: Status 404 returned error can't find the container with id f3c298ef0050cf22827f7920cc4d01a7b993f79aeb1699006930dbe36e80c572
Jul 15 21:40:34.206230 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.206209    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:40:34.367298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.367254    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:34.467794 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.467675    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:34.709433 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.709400    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" event={"ID":"40ef0e42-ca27-4edc-96cc-487a9da9446d","Type":"ContainerStarted","Data":"f3c298ef0050cf22827f7920cc4d01a7b993f79aeb1699006930dbe36e80c572"}
Jul 15 21:40:34.709612 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.709594    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:34.710895 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:34.710855    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:35.715253 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:35.715216    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" event={"ID":"40ef0e42-ca27-4edc-96cc-487a9da9446d","Type":"ContainerStarted","Data":"5186aefa18583d68307002f20a56eee626efcec0458d2f4127d155a43d726ad5"}
Jul 15 21:40:35.715621 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:35.715493    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:35.747433 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:35.747377    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" podStartSLOduration=2.212115877 podStartE2EDuration="2.74736036s" totalImagesPullingTime="535.244483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:35.745583133 +0000 UTC m=+2040.077724258" watchObservedRunningTime="2026-07-15 21:40:35.74736036 +0000 UTC m=+2040.079501450"
Jul 15 21:40:35.940821 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:35.940797    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:40:36.004642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004546    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd154804-3eed-4048-b453-a23a99c4e18b-istiod-ca-cert\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004596    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dntk\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-kube-api-access-6dntk\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004642 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004617    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-istio-token\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004667    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-socket\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004723    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-data\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004756    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-credential-socket\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004780    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-envoy\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004812    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd154804-3eed-4048-b453-a23a99c4e18b-istio-podinfo\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.004878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.004860    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-certs\") pod \"dd154804-3eed-4048-b453-a23a99c4e18b\" (UID: \"dd154804-3eed-4048-b453-a23a99c4e18b\") "
Jul 15 21:40:36.005196 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005008    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:36.005196 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005053    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:36.005196 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005102    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd154804-3eed-4048-b453-a23a99c4e18b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:40:36.005377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005239    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:36.005377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005268    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd154804-3eed-4048-b453-a23a99c4e18b-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.005377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005287    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.005377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005299    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.005786 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.005759    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:36.006886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.006860    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:40:36.006994 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.006935    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-kube-api-access-6dntk" (OuterVolumeSpecName: "kube-api-access-6dntk") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "kube-api-access-6dntk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:40:36.007095 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.007078    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:40:36.007166 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.007152    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd154804-3eed-4048-b453-a23a99c4e18b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd154804-3eed-4048-b453-a23a99c4e18b" (UID: "dd154804-3eed-4048-b453-a23a99c4e18b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:40:36.106175 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.106141    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.106175 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.106169    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.106175 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.106178    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd154804-3eed-4048-b453-a23a99c4e18b-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.106419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.106187    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd154804-3eed-4048-b453-a23a99c4e18b-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.106419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.106196    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dntk\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-kube-api-access-6dntk\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.106419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.106205    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd154804-3eed-4048-b453-a23a99c4e18b-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:40:36.722091 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.722047    2555 generic.go:356] "Generic (PLEG): container finished" podID="dd154804-3eed-4048-b453-a23a99c4e18b" containerID="389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3" exitCode=0
Jul 15 21:40:36.722569 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.722147    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"
Jul 15 21:40:36.722569 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.722158    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" event={"ID":"dd154804-3eed-4048-b453-a23a99c4e18b","Type":"ContainerDied","Data":"389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3"}
Jul 15 21:40:36.722569 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.722242    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq" event={"ID":"dd154804-3eed-4048-b453-a23a99c4e18b","Type":"ContainerDied","Data":"f0da51187ca67d30bb615e6162407ad07182db391834b678ff939fb93927fd32"}
Jul 15 21:40:36.722569 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.722275    2555 scope.go:117] "RemoveContainer" containerID="389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3"
Jul 15 21:40:36.731342 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.731330    2555 scope.go:117] "RemoveContainer" containerID="35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"
Jul 15 21:40:36.739185 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.739167    2555 scope.go:117] "RemoveContainer" containerID="389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3"
Jul 15 21:40:36.739430 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:40:36.739413    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3\": container with ID starting with 389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3 not found: ID does not exist" containerID="389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3"
Jul 15 21:40:36.739471 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.739437    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3"} err="failed to get container status \"389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3\": rpc error: code = NotFound desc = could not find container \"389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3\": container with ID starting with 389a1f767d59ef21a1793de6b1b032b70d32ab8804e3ed781db0f227ad7502d3 not found: ID does not exist"
Jul 15 21:40:36.739471 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.739452    2555 scope.go:117] "RemoveContainer" containerID="35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"
Jul 15 21:40:36.739701 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:40:36.739681    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1\": container with ID starting with 35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1 not found: ID does not exist" containerID="35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"
Jul 15 21:40:36.739757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.739706    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1"} err="failed to get container status \"35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1\": rpc error: code = NotFound desc = could not find container \"35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1\": container with ID starting with 35ed807bfe38dafc05d2f3d5fbeb0fea768b0170ca1a76745b6caf64f2226cb1 not found: ID does not exist"
Jul 15 21:40:36.747750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.747724    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"]
Jul 15 21:40:36.756112 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:36.756089    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-518fcf-openshift-default-5995f89bfc-v6ggq"]
Jul 15 21:40:38.219732 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:38.219690    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" path="/var/lib/kubelet/pods/dd154804-3eed-4048-b453-a23a99c4e18b/volumes"
Jul 15 21:40:39.180013 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:39.179976    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:39.679298 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:39.679267    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:39.891188 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:39.891127    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:40.000374 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.000287    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:40.046591 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.046549    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"]
Jul 15 21:40:40.047205 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.047177    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy"
Jul 15 21:40:40.047205 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.047205    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy"
Jul 15 21:40:40.047435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.047225    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy"
Jul 15 21:40:40.047435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.047244    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy"
Jul 15 21:40:40.047435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.047352    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy"
Jul 15 21:40:40.047435 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.047364    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd154804-3eed-4048-b453-a23a99c4e18b" containerName="istio-proxy"
Jul 15 21:40:40.049875 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.049850    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.066078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.062474    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-fd1410-openshift-default-dockercfg-29wfb\""
Jul 15 21:40:40.071399 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.071374    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"]
Jul 15 21:40:40.080277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.080247    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:40.148530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148474    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-credential-socket\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.148731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148550    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-envoy\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.148731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148624    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-socket\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.148731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148672    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-podinfo\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.148731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148702    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e061f2e-8c1f-456c-9c25-1801a3e88801-istiod-ca-cert\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.148731 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148730    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-certs\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.149056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148758    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-token\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.149056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148806    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-data\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.149056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.148833    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcxjm\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-kube-api-access-kcxjm\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.249795 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249756    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-envoy\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249825    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-socket\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249856    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-podinfo\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249874    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e061f2e-8c1f-456c-9c25-1801a3e88801-istiod-ca-cert\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249900    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-certs\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249926    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-token\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.249996    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-data\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.250022    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcxjm\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-kube-api-access-kcxjm\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250493 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.250170    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-credential-socket\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250493 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.250333    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-data\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250493 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.250442    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-socket\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.250493 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.250481    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-credential-socket\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.251120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.251096    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-certs\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.251310 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.251289    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e061f2e-8c1f-456c-9c25-1801a3e88801-istiod-ca-cert\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.252930 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.252907    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-envoy\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.253584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.253560    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-podinfo\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.265793 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.265771    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-token\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.265921 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.265894    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcxjm\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-kube-api-access-kcxjm\") pod \"e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") " pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.362997 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.362941    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:40.538329 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.536419    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"]
Jul 15 21:40:40.548608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.548574    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:40.548712 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.548640    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:40.548712 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.548667    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:40.632529 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.632459    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:40.753929 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.753885    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" event={"ID":"9e061f2e-8c1f-456c-9c25-1801a3e88801","Type":"ContainerStarted","Data":"943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"}
Jul 15 21:40:40.754360 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.753980    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" event={"ID":"9e061f2e-8c1f-456c-9c25-1801a3e88801","Type":"ContainerStarted","Data":"be5d230e596c30367136f2400ac8ee60308127b008b2c333fae655c126f17b06"}
Jul 15 21:40:40.780584 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:40.780536    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podStartSLOduration=0.780518624 podStartE2EDuration="780.518624ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:40.778549416 +0000 UTC m=+2045.110690505" watchObservedRunningTime="2026-07-15 21:40:40.780518624 +0000 UTC m=+2045.112659714"
Jul 15 21:40:41.363815 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.363779    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:41.368653 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.368630    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:41.748224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.748186    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:40:41.760094 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.760062    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:41.761214 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.761194    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:41.888069 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.888036    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"]
Jul 15 21:40:41.891186 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.891159    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:41.895382 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.895365    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-fd1410-serving-cert\""
Jul 15 21:40:41.913044 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.913016    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"]
Jul 15 21:40:41.964926 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.964894    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd9463d-2e42-4d45-92d9-cd01937480c3-openshift-service-ca\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:41.965109 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.965001    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:41.965109 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:41.965039    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbrs\" (UniqueName: \"kubernetes.io/projected/8fd9463d-2e42-4d45-92d9-cd01937480c3-kube-api-access-4jbrs\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.066534 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.066503    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.066660 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.066569    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbrs\" (UniqueName: \"kubernetes.io/projected/8fd9463d-2e42-4d45-92d9-cd01937480c3-kube-api-access-4jbrs\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.066660 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.066639    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd9463d-2e42-4d45-92d9-cd01937480c3-openshift-service-ca\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.066761 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:40:42.066663    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-serving-cert: secret "maas-api-e2e-api-b-fd1410-serving-cert" not found
Jul 15 21:40:42.066825 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:40:42.066772    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls podName:8fd9463d-2e42-4d45-92d9-cd01937480c3 nodeName:}" failed. No retries permitted until 2026-07-15 21:40:42.566748034 +0000 UTC m=+2046.898889104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls") pod "maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" (UID: "8fd9463d-2e42-4d45-92d9-cd01937480c3") : secret "maas-api-e2e-api-b-fd1410-serving-cert" not found
Jul 15 21:40:42.067345 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.067327    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd9463d-2e42-4d45-92d9-cd01937480c3-openshift-service-ca\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.077514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.077491    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbrs\" (UniqueName: \"kubernetes.io/projected/8fd9463d-2e42-4d45-92d9-cd01937480c3-kube-api-access-4jbrs\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.183965 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.183914    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:42.289183 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.289154    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:42.359202 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.359119    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:42.571280 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.571245    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.573782 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.573749    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls\") pod \"maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.629499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.629465    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:42.726267 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.726230    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:42.804151 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.804119    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:42.947815 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:42.947774    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"]
Jul 15 21:40:42.950975 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:40:42.950930    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd9463d_2e42_4d45_92d9_cd01937480c3.slice/crio-5eb9074756dbb66662c83dc4a65327486c8ffc4e45abc7a185efe84017a0e1d0 WatchSource:0}: Error finding container 5eb9074756dbb66662c83dc4a65327486c8ffc4e45abc7a185efe84017a0e1d0: Status 404 returned error can't find the container with id 5eb9074756dbb66662c83dc4a65327486c8ffc4e45abc7a185efe84017a0e1d0
Jul 15 21:40:43.772134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:43.772047    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" event={"ID":"8fd9463d-2e42-4d45-92d9-cd01937480c3","Type":"ContainerStarted","Data":"5eb9074756dbb66662c83dc4a65327486c8ffc4e45abc7a185efe84017a0e1d0"}
Jul 15 21:40:44.778839 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.778805    2555 generic.go:356] "Generic (PLEG): container finished" podID="2500a58d-b58c-498e-968b-9fd6e3618678" containerID="f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692" exitCode=0
Jul 15 21:40:44.779287 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.778881    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" event={"ID":"2500a58d-b58c-498e-968b-9fd6e3618678","Type":"ContainerDied","Data":"f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"}
Jul 15 21:40:44.779412 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.779396    2555 scope.go:117] "RemoveContainer" containerID="f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"
Jul 15 21:40:44.780482 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.780453    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" event={"ID":"8fd9463d-2e42-4d45-92d9-cd01937480c3","Type":"ContainerStarted","Data":"53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e"}
Jul 15 21:40:44.780608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.780476    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:44.780608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.780544    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:44.780608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.780576    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:44.780608 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.780586    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:44.839414 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:44.839358    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" podStartSLOduration=2.784546697 podStartE2EDuration="3.839337101s" totalImagesPullingTime="1.054790404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:44.838469112 +0000 UTC m=+2049.170610201" watchObservedRunningTime="2026-07-15 21:40:44.839337101 +0000 UTC m=+2049.171478191"
Jul 15 21:40:45.788828 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:45.788799    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" event={"ID":"2500a58d-b58c-498e-968b-9fd6e3618678","Type":"ContainerStarted","Data":"ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1"}
Jul 15 21:40:45.902558 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:45.902527    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:45.903799 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:45.903764    2555 patch_prober.go:28] interesting pod/e2e-api-a-47a933-openshift-default-56fd85d589-267vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 15 21:40:45.903933 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:45.903828    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 15 21:40:46.903150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:46.903118    2555 patch_prober.go:28] interesting pod/e2e-api-a-47a933-openshift-default-56fd85d589-267vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 15 21:40:46.903540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:46.903184    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 15 21:40:47.280233 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:47.280138    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:47.902821 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:47.902781    2555 patch_prober.go:28] interesting pod/e2e-api-a-47a933-openshift-default-56fd85d589-267vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 15 21:40:47.903117 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:47.902849    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 15 21:40:48.903278 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:48.903248    2555 patch_prober.go:28] interesting pod/e2e-api-a-47a933-openshift-default-56fd85d589-267vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 15 21:40:48.903644 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:48.903299    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 15 21:40:49.710379 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:49.710348    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:49.906505 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:49.906473    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:49.907676 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:49.907649    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:40:50.794242 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:50.794214    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:40:51.074771 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:51.074696    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:51.175425 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:51.175389    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:55.182218 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:55.182178    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:55.282499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:55.282464    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:55.769932 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:55.769898    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:40:56.761951 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.761913    2555 patch_prober.go:28] interesting pod/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 15 21:40:56.762366 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.761996    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 15 21:40:56.859059 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.859027    2555 generic.go:356] "Generic (PLEG): container finished" podID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerID="943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1" exitCode=0
Jul 15 21:40:56.859224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.859099    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" event={"ID":"9e061f2e-8c1f-456c-9c25-1801a3e88801","Type":"ContainerDied","Data":"943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"}
Jul 15 21:40:56.859508 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.859495    2555 scope.go:117] "RemoveContainer" containerID="943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"
Jul 15 21:40:56.860526 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.860501    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:56.860586 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.860567    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:56.860623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:56.860599    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:40:57.867100 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:57.867019    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" event={"ID":"9e061f2e-8c1f-456c-9c25-1801a3e88801","Type":"ContainerStarted","Data":"c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e"}
Jul 15 21:40:58.363818 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:58.363785    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:40:58.364061 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:58.364040    2555 patch_prober.go:28] interesting pod/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 15 21:40:58.364126 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:58.364088    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 15 21:40:59.364334 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:59.364292    2555 patch_prober.go:28] interesting pod/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 15 21:40:59.364745 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:40:59.364347    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 15 21:41:00.363761 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:00.363726    2555 patch_prober.go:28] interesting pod/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 15 21:41:00.363941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:00.363781    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 15 21:41:01.363372 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.363328    2555 patch_prober.go:28] interesting pod/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 15 21:41:01.363773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.363388    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 15 21:41:01.597385 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.597346    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"]
Jul 15 21:41:01.597852 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.597817    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" podUID="40ef0e42-ca27-4edc-96cc-487a9da9446d" containerName="maas-api" containerID="cri-o://5186aefa18583d68307002f20a56eee626efcec0458d2f4127d155a43d726ad5" gracePeriod=30
Jul 15 21:41:01.646978 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.643427    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:01.724979 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.723600    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" podUID="40ef0e42-ca27-4edc-96cc-487a9da9446d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.55:8443/health\": dial tcp 10.134.0.55:8443: connect: connection refused"
Jul 15 21:41:01.888017 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.887990    2555 generic.go:356] "Generic (PLEG): container finished" podID="40ef0e42-ca27-4edc-96cc-487a9da9446d" containerID="5186aefa18583d68307002f20a56eee626efcec0458d2f4127d155a43d726ad5" exitCode=0
Jul 15 21:41:01.888141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.888058    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" event={"ID":"40ef0e42-ca27-4edc-96cc-487a9da9446d","Type":"ContainerDied","Data":"5186aefa18583d68307002f20a56eee626efcec0458d2f4127d155a43d726ad5"}
Jul 15 21:41:01.888141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.888108    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg" event={"ID":"40ef0e42-ca27-4edc-96cc-487a9da9446d","Type":"ContainerDied","Data":"f3c298ef0050cf22827f7920cc4d01a7b993f79aeb1699006930dbe36e80c572"}
Jul 15 21:41:01.888141 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.888127    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3c298ef0050cf22827f7920cc4d01a7b993f79aeb1699006930dbe36e80c572"
Jul 15 21:41:01.898850 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:01.898791    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:41:02.069146 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.069115    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ef0e42-ca27-4edc-96cc-487a9da9446d-maas-api-tls\") pod \"40ef0e42-ca27-4edc-96cc-487a9da9446d\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") "
Jul 15 21:41:02.069303 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.069162    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-756h4\" (UniqueName: \"kubernetes.io/projected/40ef0e42-ca27-4edc-96cc-487a9da9446d-kube-api-access-756h4\") pod \"40ef0e42-ca27-4edc-96cc-487a9da9446d\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") "
Jul 15 21:41:02.069303 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.069222    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ef0e42-ca27-4edc-96cc-487a9da9446d-openshift-service-ca\") pod \"40ef0e42-ca27-4edc-96cc-487a9da9446d\" (UID: \"40ef0e42-ca27-4edc-96cc-487a9da9446d\") "
Jul 15 21:41:02.069617 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.069594    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40ef0e42-ca27-4edc-96cc-487a9da9446d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40ef0e42-ca27-4edc-96cc-487a9da9446d" (UID: "40ef0e42-ca27-4edc-96cc-487a9da9446d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:41:02.071334 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.071300    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40ef0e42-ca27-4edc-96cc-487a9da9446d-kube-api-access-756h4" (OuterVolumeSpecName: "kube-api-access-756h4") pod "40ef0e42-ca27-4edc-96cc-487a9da9446d" (UID: "40ef0e42-ca27-4edc-96cc-487a9da9446d"). InnerVolumeSpecName "kube-api-access-756h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:02.071450 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.071363    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40ef0e42-ca27-4edc-96cc-487a9da9446d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40ef0e42-ca27-4edc-96cc-487a9da9446d" (UID: "40ef0e42-ca27-4edc-96cc-487a9da9446d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:41:02.170602 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.170522    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40ef0e42-ca27-4edc-96cc-487a9da9446d-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:02.170602 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.170556    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-756h4\" (UniqueName: \"kubernetes.io/projected/40ef0e42-ca27-4edc-96cc-487a9da9446d-kube-api-access-756h4\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:02.170602 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.170568    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40ef0e42-ca27-4edc-96cc-487a9da9446d-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:02.367797 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.367768    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:41:02.368235 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.368054    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:41:02.368796 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.368780    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:41:02.624468 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.624414    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:02.892750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.892662    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"
Jul 15 21:41:02.913329 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.913299    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"]
Jul 15 21:41:02.919443 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:02.919414    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-47a933-7f9456b84c-v4vjg"]
Jul 15 21:41:03.519605 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:03.519572    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:03.619397 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:03.619354    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:04.217135 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:04.217104    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40ef0e42-ca27-4edc-96cc-487a9da9446d" path="/var/lib/kubelet/pods/40ef0e42-ca27-4edc-96cc-487a9da9446d/volumes"
Jul 15 21:41:08.922885 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:08.922846    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:09.024835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:09.024791    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:09.245531 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:09.245452    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"]
Jul 15 21:41:09.245718 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:09.245659    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy" containerID="cri-o://ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1" gracePeriod=30
Jul 15 21:41:09.263250 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:09.263204    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:09.458107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:09.458065    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:09.932532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:09.932503    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:14.523036 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.523009    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:41:14.577344 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577317    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-certs\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577352    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-data\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577372    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2500a58d-b58c-498e-968b-9fd6e3618678-istio-podinfo\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577432    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsz6g\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-kube-api-access-lsz6g\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577486    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2500a58d-b58c-498e-968b-9fd6e3618678-istiod-ca-cert\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577506    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-envoy\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577582    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-istio-token\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577628    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-socket\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.577790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577639    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:14.577790 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577652    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-credential-socket\") pod \"2500a58d-b58c-498e-968b-9fd6e3618678\" (UID: \"2500a58d-b58c-498e-968b-9fd6e3618678\") "
Jul 15 21:41:14.578050 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577932    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2500a58d-b58c-498e-968b-9fd6e3618678-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:41:14.578050 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.577940    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.578180 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.578151    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:14.578180 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.578170    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:14.578477 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.578449    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-data" (OuterVolumeSpecName: "istio-data") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:14.579729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.579700    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2500a58d-b58c-498e-968b-9fd6e3618678-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:41:14.579830 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.579741    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-istio-token" (OuterVolumeSpecName: "istio-token") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:14.579830 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.579785    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:14.580591 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.580566    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-kube-api-access-lsz6g" (OuterVolumeSpecName: "kube-api-access-lsz6g") pod "2500a58d-b58c-498e-968b-9fd6e3618678" (UID: "2500a58d-b58c-498e-968b-9fd6e3618678"). InnerVolumeSpecName "kube-api-access-lsz6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:14.678418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678385    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678414    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678418 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678424    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678434    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2500a58d-b58c-498e-968b-9fd6e3618678-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678443    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsz6g\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-kube-api-access-lsz6g\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678451    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2500a58d-b58c-498e-968b-9fd6e3618678-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678459    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2500a58d-b58c-498e-968b-9fd6e3618678-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.678703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.678467    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2500a58d-b58c-498e-968b-9fd6e3618678-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:14.955533 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.955490    2555 generic.go:356] "Generic (PLEG): container finished" podID="2500a58d-b58c-498e-968b-9fd6e3618678" containerID="ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1" exitCode=0
Jul 15 21:41:14.955708 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.955533    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" event={"ID":"2500a58d-b58c-498e-968b-9fd6e3618678","Type":"ContainerDied","Data":"ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1"}
Jul 15 21:41:14.955708 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.955547    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"
Jul 15 21:41:14.955708 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.955594    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh" event={"ID":"2500a58d-b58c-498e-968b-9fd6e3618678","Type":"ContainerDied","Data":"38ccfb075c63e0088f7282791c6de3d15dd596ec864059da0210623f8940a22f"}
Jul 15 21:41:14.955708 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.955617    2555 scope.go:117] "RemoveContainer" containerID="ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1"
Jul 15 21:41:14.964782 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.964764    2555 scope.go:117] "RemoveContainer" containerID="f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"
Jul 15 21:41:14.972533 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.972517    2555 scope.go:117] "RemoveContainer" containerID="ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1"
Jul 15 21:41:14.972815 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:14.972795    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1\": container with ID starting with ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1 not found: ID does not exist" containerID="ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1"
Jul 15 21:41:14.972862 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.972823    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1"} err="failed to get container status \"ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1\": rpc error: code = NotFound desc = could not find container \"ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1\": container with ID starting with ed0941d8efd101ed0d341ca12a4a6d11b43c13234fd947ada90cacb674d3fbd1 not found: ID does not exist"
Jul 15 21:41:14.972862 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.972843    2555 scope.go:117] "RemoveContainer" containerID="f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"
Jul 15 21:41:14.973162 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:14.973145    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692\": container with ID starting with f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692 not found: ID does not exist" containerID="f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"
Jul 15 21:41:14.973219 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.973167    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692"} err="failed to get container status \"f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692\": rpc error: code = NotFound desc = could not find container \"f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692\": container with ID starting with f77380c7083f5d8dd7bea719098d06c54070410d93d752ef9ee6d4a7f0127692 not found: ID does not exist"
Jul 15 21:41:14.988553 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.988532    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"]
Jul 15 21:41:14.999030 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:14.999004    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-47a933-openshift-default-56fd85d589-267vh"]
Jul 15 21:41:15.017420 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.017391    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"]
Jul 15 21:41:15.017673 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.017630    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" podUID="8fd9463d-2e42-4d45-92d9-cd01937480c3" containerName="maas-api" containerID="cri-o://53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e" gracePeriod=30
Jul 15 21:41:15.058097 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.058050    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:15.255967 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.255927    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:41:15.282881 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.282854    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd9463d-2e42-4d45-92d9-cd01937480c3-openshift-service-ca\") pod \"8fd9463d-2e42-4d45-92d9-cd01937480c3\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") "
Jul 15 21:41:15.283027 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.282907    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls\") pod \"8fd9463d-2e42-4d45-92d9-cd01937480c3\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") "
Jul 15 21:41:15.283027 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.282988    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jbrs\" (UniqueName: \"kubernetes.io/projected/8fd9463d-2e42-4d45-92d9-cd01937480c3-kube-api-access-4jbrs\") pod \"8fd9463d-2e42-4d45-92d9-cd01937480c3\" (UID: \"8fd9463d-2e42-4d45-92d9-cd01937480c3\") "
Jul 15 21:41:15.283224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.283198    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fd9463d-2e42-4d45-92d9-cd01937480c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8fd9463d-2e42-4d45-92d9-cd01937480c3" (UID: "8fd9463d-2e42-4d45-92d9-cd01937480c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:41:15.283398 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.283377    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd9463d-2e42-4d45-92d9-cd01937480c3-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:15.284945 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.284919    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8fd9463d-2e42-4d45-92d9-cd01937480c3" (UID: "8fd9463d-2e42-4d45-92d9-cd01937480c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:41:15.284945 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.284980    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd9463d-2e42-4d45-92d9-cd01937480c3-kube-api-access-4jbrs" (OuterVolumeSpecName: "kube-api-access-4jbrs") pod "8fd9463d-2e42-4d45-92d9-cd01937480c3" (UID: "8fd9463d-2e42-4d45-92d9-cd01937480c3"). InnerVolumeSpecName "kube-api-access-4jbrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:15.384296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.384265    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jbrs\" (UniqueName: \"kubernetes.io/projected/8fd9463d-2e42-4d45-92d9-cd01937480c3-kube-api-access-4jbrs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:15.384296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.384289    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd9463d-2e42-4d45-92d9-cd01937480c3-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:15.963290 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.963258    2555 generic.go:356] "Generic (PLEG): container finished" podID="8fd9463d-2e42-4d45-92d9-cd01937480c3" containerID="53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e" exitCode=0
Jul 15 21:41:15.963729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.963304    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"
Jul 15 21:41:15.963729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.963332    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" event={"ID":"8fd9463d-2e42-4d45-92d9-cd01937480c3","Type":"ContainerDied","Data":"53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e"}
Jul 15 21:41:15.963729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.963385    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f" event={"ID":"8fd9463d-2e42-4d45-92d9-cd01937480c3","Type":"ContainerDied","Data":"5eb9074756dbb66662c83dc4a65327486c8ffc4e45abc7a185efe84017a0e1d0"}
Jul 15 21:41:15.963729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.963418    2555 scope.go:117] "RemoveContainer" containerID="53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e"
Jul 15 21:41:15.972567 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.972552    2555 scope.go:117] "RemoveContainer" containerID="53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e"
Jul 15 21:41:15.972791 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:15.972772    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e\": container with ID starting with 53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e not found: ID does not exist" containerID="53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e"
Jul 15 21:41:15.972853 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.972798    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e"} err="failed to get container status \"53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e\": rpc error: code = NotFound desc = could not find container \"53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e\": container with ID starting with 53a0a5fa12890f5d6a6bbd788db3c252208bb2af64415cb2d5b31e0ada33ed6e not found: ID does not exist"
Jul 15 21:41:15.988499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.988470    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"]
Jul 15 21:41:15.990922 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:15.990901    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fd1410-5dcb94bf4c-pxz4f"]
Jul 15 21:41:16.218067 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:16.217989    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" path="/var/lib/kubelet/pods/2500a58d-b58c-498e-968b-9fd6e3618678/volumes"
Jul 15 21:41:16.218536 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:16.218517    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fd9463d-2e42-4d45-92d9-cd01937480c3" path="/var/lib/kubelet/pods/8fd9463d-2e42-4d45-92d9-cd01937480c3/volumes"
Jul 15 21:41:16.365655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:16.365621    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:17.443388 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:17.443345    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:23.237655 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:23.237621    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:23.338249 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:23.338210    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:23.499676 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:23.499600    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"]
Jul 15 21:41:23.499831 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:23.499801    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy" containerID="cri-o://c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e" gracePeriod=30
Jul 15 21:41:23.509942 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:23.509918    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:23.765031 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:23.764918    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:24.125134 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.125093    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:24.433699 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.433607    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"]
Jul 15 21:41:24.434197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434177    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy"
Jul 15 21:41:24.434197 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434196    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy"
Jul 15 21:41:24.434311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434207    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40ef0e42-ca27-4edc-96cc-487a9da9446d" containerName="maas-api"
Jul 15 21:41:24.434311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434215    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="40ef0e42-ca27-4edc-96cc-487a9da9446d" containerName="maas-api"
Jul 15 21:41:24.434311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434227    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy"
Jul 15 21:41:24.434311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434235    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy"
Jul 15 21:41:24.434311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434242    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fd9463d-2e42-4d45-92d9-cd01937480c3" containerName="maas-api"
Jul 15 21:41:24.434311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434251    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd9463d-2e42-4d45-92d9-cd01937480c3" containerName="maas-api"
Jul 15 21:41:24.434589 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434374    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy"
Jul 15 21:41:24.434589 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434387    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fd9463d-2e42-4d45-92d9-cd01937480c3" containerName="maas-api"
Jul 15 21:41:24.434589 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.434401    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="40ef0e42-ca27-4edc-96cc-487a9da9446d" containerName="maas-api"
Jul 15 21:41:24.436698 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.436677    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.439544 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.439525    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-fa020c-openshift-default-dockercfg-cg8tx\""
Jul 15 21:41:24.452309 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.452241    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"]
Jul 15 21:41:24.463451 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.463410    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:24.563339 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563308    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-certs\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563339 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563346    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-token\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563367    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdv4z\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-kube-api-access-bdv4z\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563432    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-socket\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563465    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-envoy\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563480    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-podinfo\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563507    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istiod-ca-cert\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563533    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-credential-socket\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.563556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.563557    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-data\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664176 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664137    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-token\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664185    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdv4z\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-kube-api-access-bdv4z\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664239    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-socket\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664293    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-envoy\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664386 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664318    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-podinfo\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664614 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664477    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istiod-ca-cert\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664614 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664539    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-credential-socket\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664614 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664567    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-data\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664763 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664617    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-certs\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.664998 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.664936    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-certs\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.665131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.665084    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-credential-socket\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.665200 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.665138    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-data\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.665377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.665352    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istiod-ca-cert\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.665443 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.665386    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-socket\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.667271 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.667237    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-podinfo\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.667271 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.667259    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-envoy\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.678304 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.678281    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdv4z\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-kube-api-access-bdv4z\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.678421 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.678307    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-token\") pod \"e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") " pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.738445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.738366    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:24.748381 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.748354    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:24.895339 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.895293    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"]
Jul 15 21:41:24.897595 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:41:24.897568    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc0ab2d_4fe8_4608_9d82_fcfdfe87f516.slice/crio-d7910394f398d4c304d5f3fbcec605ef01ad20aa53cc239cc31952a626992c5f WatchSource:0}: Error finding container d7910394f398d4c304d5f3fbcec605ef01ad20aa53cc239cc31952a626992c5f: Status 404 returned error can't find the container with id d7910394f398d4c304d5f3fbcec605ef01ad20aa53cc239cc31952a626992c5f
Jul 15 21:41:24.899623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.899593    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:24.899682 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.899654    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:24.899722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:24.899682    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:25.014700 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:25.014598    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" event={"ID":"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516","Type":"ContainerStarted","Data":"a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"}
Jul 15 21:41:25.014700 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:25.014653    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" event={"ID":"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516","Type":"ContainerStarted","Data":"d7910394f398d4c304d5f3fbcec605ef01ad20aa53cc239cc31952a626992c5f"}
Jul 15 21:41:25.039273 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:25.039207    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podStartSLOduration=1.039184272 podStartE2EDuration="1.039184272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:41:25.038471656 +0000 UTC m=+2089.370612772" watchObservedRunningTime="2026-07-15 21:41:25.039184272 +0000 UTC m=+2089.371325387"
Jul 15 21:41:25.748726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:25.748688    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:25.753847 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:25.753817    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:26.017284 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.017177    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:26.023893 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.023868    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:26.025008 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.024990    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:26.098194 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.098162    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:26.239559 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.239533    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"]
Jul 15 21:41:26.239933 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.239922    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2500a58d-b58c-498e-968b-9fd6e3618678" containerName="istio-proxy"
Jul 15 21:41:26.242305 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.242288    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.246771 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.246749    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-fa020c-serving-cert\""
Jul 15 21:41:26.246902 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.246885    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xsfh4\""
Jul 15 21:41:26.257317 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.257298    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"]
Jul 15 21:41:26.279106 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.279010    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.279300 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.279163    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzkc2\" (UniqueName: \"kubernetes.io/projected/dee527f2-0282-452e-a713-7fb8c51cbaa8-kube-api-access-wzkc2\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.279300 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.279231    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee527f2-0282-452e-a713-7fb8c51cbaa8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.286597 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.286577    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:26.380295 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.380268    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzkc2\" (UniqueName: \"kubernetes.io/projected/dee527f2-0282-452e-a713-7fb8c51cbaa8-kube-api-access-wzkc2\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.380463 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.380314    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee527f2-0282-452e-a713-7fb8c51cbaa8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.380463 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.380342    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.380553 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:26.380466    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-serving-cert: secret "maas-api-e2e-shared-a-fa020c-serving-cert" not found
Jul 15 21:41:26.380553 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:26.380520    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls podName:dee527f2-0282-452e-a713-7fb8c51cbaa8 nodeName:}" failed. No retries permitted until 2026-07-15 21:41:26.880502393 +0000 UTC m=+2091.212643464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls") pod "maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" (UID: "dee527f2-0282-452e-a713-7fb8c51cbaa8") : secret "maas-api-e2e-shared-a-fa020c-serving-cert" not found
Jul 15 21:41:26.380998 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.380979    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee527f2-0282-452e-a713-7fb8c51cbaa8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.392679 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.392654    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzkc2\" (UniqueName: \"kubernetes.io/projected/dee527f2-0282-452e-a713-7fb8c51cbaa8-kube-api-access-wzkc2\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.513260 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.513228    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:26.792687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.792657    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:26.885507 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.885474    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.887886 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.887866    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-fa020c-689679759c-hnjsf\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:26.893700 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:26.893675    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:27.152757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:27.152724    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:27.287463 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:27.287433    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"]
Jul 15 21:41:27.288558 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:41:27.288519    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee527f2_0282_452e_a713_7fb8c51cbaa8.slice/crio-a50ee3352c5f816438f44c6af8972a4c068fa2030fb1044a1cd78aa0ed1d0081 WatchSource:0}: Error finding container a50ee3352c5f816438f44c6af8972a4c068fa2030fb1044a1cd78aa0ed1d0081: Status 404 returned error can't find the container with id a50ee3352c5f816438f44c6af8972a4c068fa2030fb1044a1cd78aa0ed1d0081
Jul 15 21:41:27.848888 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:27.848861    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:28.037876 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.037850    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" event={"ID":"dee527f2-0282-452e-a713-7fb8c51cbaa8","Type":"ContainerStarted","Data":"a50ee3352c5f816438f44c6af8972a4c068fa2030fb1044a1cd78aa0ed1d0081"}
Jul 15 21:41:28.732178 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.732156    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:41:28.802800 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802724    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-credential-socket\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.802800 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802772    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-podinfo\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.802800 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802797    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-certs\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802824    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcxjm\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-kube-api-access-kcxjm\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802851    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-token\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802870    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e061f2e-8c1f-456c-9c25-1801a3e88801-istiod-ca-cert\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802900    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-socket\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.802986    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-envoy\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803033    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-data\") pod \"9e061f2e-8c1f-456c-9c25-1801a3e88801\" (UID: \"9e061f2e-8c1f-456c-9c25-1801a3e88801\") "
Jul 15 21:41:28.803078 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803046    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:28.803449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803138    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:28.803449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803275    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:28.803449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803375    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.803449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803394    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.803449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803434    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.803702 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.803583    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e061f2e-8c1f-456c-9c25-1801a3e88801-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:41:28.804285 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.804257    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:28.804978 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.804943    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:41:28.805088 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.805010    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:28.805196 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.805178    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:41:28.805249 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.805182    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-kube-api-access-kcxjm" (OuterVolumeSpecName: "kube-api-access-kcxjm") pod "9e061f2e-8c1f-456c-9c25-1801a3e88801" (UID: "9e061f2e-8c1f-456c-9c25-1801a3e88801"). InnerVolumeSpecName "kube-api-access-kcxjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:28.903903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.903870    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.903903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.903895    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.903903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.903904    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.904358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.903916    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcxjm\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-kube-api-access-kcxjm\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.904358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.903926    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e061f2e-8c1f-456c-9c25-1801a3e88801-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:28.904358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:28.903935    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e061f2e-8c1f-456c-9c25-1801a3e88801-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:41:29.044981 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.044921    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" event={"ID":"dee527f2-0282-452e-a713-7fb8c51cbaa8","Type":"ContainerStarted","Data":"becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d"}
Jul 15 21:41:29.045186 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.045091    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:29.046624 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.046596    2555 generic.go:356] "Generic (PLEG): container finished" podID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerID="c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e" exitCode=0
Jul 15 21:41:29.046750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.046636    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"
Jul 15 21:41:29.046750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.046680    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" event={"ID":"9e061f2e-8c1f-456c-9c25-1801a3e88801","Type":"ContainerDied","Data":"c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e"}
Jul 15 21:41:29.046750 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.046736    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4" event={"ID":"9e061f2e-8c1f-456c-9c25-1801a3e88801","Type":"ContainerDied","Data":"be5d230e596c30367136f2400ac8ee60308127b008b2c333fae655c126f17b06"}
Jul 15 21:41:29.046869 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.046767    2555 scope.go:117] "RemoveContainer" containerID="c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e"
Jul 15 21:41:29.056894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.056871    2555 scope.go:117] "RemoveContainer" containerID="943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"
Jul 15 21:41:29.065224 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.065158    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" podStartSLOduration=2.3592787619999998 podStartE2EDuration="3.06514704s" totalImagesPullingTime="705.868278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:41:29.064603378 +0000 UTC m=+2093.396744522" watchObservedRunningTime="2026-07-15 21:41:29.06514704 +0000 UTC m=+2093.397288129"
Jul 15 21:41:29.065306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.065241    2555 scope.go:117] "RemoveContainer" containerID="c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e"
Jul 15 21:41:29.065565 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:29.065542    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e\": container with ID starting with c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e not found: ID does not exist" containerID="c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e"
Jul 15 21:41:29.065633 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.065571    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e"} err="failed to get container status \"c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e\": rpc error: code = NotFound desc = could not find container \"c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e\": container with ID starting with c90f72f15f6911201ac5f9ed12d6afedba901da825cde63090ab755a698a028e not found: ID does not exist"
Jul 15 21:41:29.065633 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.065588    2555 scope.go:117] "RemoveContainer" containerID="943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"
Jul 15 21:41:29.065841 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:41:29.065826    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1\": container with ID starting with 943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1 not found: ID does not exist" containerID="943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"
Jul 15 21:41:29.065880 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.065844    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1"} err="failed to get container status \"943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1\": rpc error: code = NotFound desc = could not find container \"943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1\": container with ID starting with 943104ad618d5b7e0f8c8202f1ac5170389761224b8aae31a53c233f8fe8c1d1 not found: ID does not exist"
Jul 15 21:41:29.080359 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.080328    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"]
Jul 15 21:41:29.085945 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:29.085926    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-fd1410-openshift-default-7f9cbc5b9-qhvg4"]
Jul 15 21:41:30.218240 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:30.218206    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" path="/var/lib/kubelet/pods/9e061f2e-8c1f-456c-9c25-1801a3e88801/volumes"
Jul 15 21:41:31.511869 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:31.511832    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:31.893559 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:31.893514    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:32.306571 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.306482    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"]
Jul 15 21:41:32.307035 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.307020    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy"
Jul 15 21:41:32.307084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.307038    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy"
Jul 15 21:41:32.307084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.307058    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy"
Jul 15 21:41:32.307084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.307068    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy"
Jul 15 21:41:32.307190 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.307164    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy"
Jul 15 21:41:32.310667 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.310643    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.314130 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.314105    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-7ef33e-openshift-default-dockercfg-pkrxm\""
Jul 15 21:41:32.336384 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.334270    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"]
Jul 15 21:41:32.337356 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.337326    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:32.450262 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450232    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-certs\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450437 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450385    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db6b571-908f-4553-b36e-c45d56ec70e2-istiod-ca-cert\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450437 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450431    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-podinfo\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450531 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450474    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-socket\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450531 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450496    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-token\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450641 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450547    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-credential-socket\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450691 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450657    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-envoy\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450709    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-data\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.450798 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.450742    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5gjh\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-kube-api-access-n5gjh\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551313 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551269    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db6b571-908f-4553-b36e-c45d56ec70e2-istiod-ca-cert\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551313 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551316    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-podinfo\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551498    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-socket\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551543    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-token\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551589    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-credential-socket\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551659    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-envoy\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551700    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-data\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551732    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5gjh\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-kube-api-access-n5gjh\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551772    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-certs\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.551857 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.551847    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-socket\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.552323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.552009    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-credential-socket\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.552323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.552073    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db6b571-908f-4553-b36e-c45d56ec70e2-istiod-ca-cert\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.552323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.552148    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-certs\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.552323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.552256    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-data\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.554047 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.554024    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-envoy\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.554146 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.554065    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-podinfo\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.567999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.567904    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-token\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.568102 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.568015    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5gjh\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-kube-api-access-n5gjh\") pod \"e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") " pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.639061 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.639025    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:32.805788 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.805763    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"]
Jul 15 21:41:32.808023 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:41:32.807998    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db6b571_908f_4553_b36e_c45d56ec70e2.slice/crio-0f6f585fa564ea0b35b09ce188972ca48f92a1fca41ea7c394d6a83d913cd1c7 WatchSource:0}: Error finding container 0f6f585fa564ea0b35b09ce188972ca48f92a1fca41ea7c394d6a83d913cd1c7: Status 404 returned error can't find the container with id 0f6f585fa564ea0b35b09ce188972ca48f92a1fca41ea7c394d6a83d913cd1c7
Jul 15 21:41:32.810685 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.810655    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:32.810755 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.810733    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:32.810815 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:32.810768    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:33.068145 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:33.068102    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" event={"ID":"6db6b571-908f-4553-b36e-c45d56ec70e2","Type":"ContainerStarted","Data":"30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"}
Jul 15 21:41:33.068312 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:33.068162    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" event={"ID":"6db6b571-908f-4553-b36e-c45d56ec70e2","Type":"ContainerStarted","Data":"0f6f585fa564ea0b35b09ce188972ca48f92a1fca41ea7c394d6a83d913cd1c7"}
Jul 15 21:41:33.091710 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:33.091608    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" podStartSLOduration=1.091587925 podStartE2EDuration="1.091587925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:41:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:41:33.089183231 +0000 UTC m=+2097.421324323" watchObservedRunningTime="2026-07-15 21:41:33.091587925 +0000 UTC m=+2097.423729016"
Jul 15 21:41:33.243290 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:33.243255    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:33.640202 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:33.640167    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:33.645554 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:33.645524    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:34.073810 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.073735    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:34.074723 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.074700    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:34.090003 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.089944    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:34.186728 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.186693    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"]
Jul 15 21:41:34.187156 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.187106    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:34.187442 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.187421    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e061f2e-8c1f-456c-9c25-1801a3e88801" containerName="istio-proxy"
Jul 15 21:41:34.190192 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.190169    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.192624 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.192598    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-7ef33e-serving-cert\""
Jul 15 21:41:34.201651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.201628    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"]
Jul 15 21:41:34.265610 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.265576    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f86d951-eba7-4b69-abfa-fc2469fa827f-maas-api-tls\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.265610 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.265615    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr8cx\" (UniqueName: \"kubernetes.io/projected/9f86d951-eba7-4b69-abfa-fc2469fa827f-kube-api-access-xr8cx\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.265811 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.265642    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f86d951-eba7-4b69-abfa-fc2469fa827f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.278703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.278671    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:34.367323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.367227    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f86d951-eba7-4b69-abfa-fc2469fa827f-maas-api-tls\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.367323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.367275    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr8cx\" (UniqueName: \"kubernetes.io/projected/9f86d951-eba7-4b69-abfa-fc2469fa827f-kube-api-access-xr8cx\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.367556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.367409    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f86d951-eba7-4b69-abfa-fc2469fa827f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.368050 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.368026    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f86d951-eba7-4b69-abfa-fc2469fa827f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.369777 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.369754    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f86d951-eba7-4b69-abfa-fc2469fa827f-maas-api-tls\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.370088 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.370060    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:34.385545 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.385516    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr8cx\" (UniqueName: \"kubernetes.io/projected/9f86d951-eba7-4b69-abfa-fc2469fa827f-kube-api-access-xr8cx\") pod \"maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.503733 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.503702    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:34.643722 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.643690    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"]
Jul 15 21:41:34.644130 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:41:34.643998    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f86d951_eba7_4b69_abfa_fc2469fa827f.slice/crio-7f0c1c2449e628daa40a24eb3efb5b69576cfaf8755cf4f41208da39f6a1c826 WatchSource:0}: Error finding container 7f0c1c2449e628daa40a24eb3efb5b69576cfaf8755cf4f41208da39f6a1c826: Status 404 returned error can't find the container with id 7f0c1c2449e628daa40a24eb3efb5b69576cfaf8755cf4f41208da39f6a1c826
Jul 15 21:41:34.904995 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:34.904886    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:35.009282 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:35.009248    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:35.058757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:35.058730    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:41:35.080222 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:35.080188    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw" event={"ID":"9f86d951-eba7-4b69-abfa-fc2469fa827f","Type":"ContainerStarted","Data":"7f0c1c2449e628daa40a24eb3efb5b69576cfaf8755cf4f41208da39f6a1c826"}
Jul 15 21:41:36.087014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:36.086979    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw" event={"ID":"9f86d951-eba7-4b69-abfa-fc2469fa827f","Type":"ContainerStarted","Data":"7fe8a366e1e36e21c74423966c05370ff9dc0260847547853876a03d9e0fbcdf"}
Jul 15 21:41:36.087488 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:36.087205    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:36.106499 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:36.106457    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw" podStartSLOduration=1.61291722 podStartE2EDuration="2.106444585s" totalImagesPullingTime="493.527365ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:41:36.10630658 +0000 UTC m=+2100.438447696" watchObservedRunningTime="2026-07-15 21:41:36.106444585 +0000 UTC m=+2100.438585675"
Jul 15 21:41:36.250327 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:36.250299    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:41:36.253946 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:36.253918    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:41:38.098034 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:38.098002    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerID="a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead" exitCode=0
Jul 15 21:41:38.098439 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:38.098062    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" event={"ID":"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516","Type":"ContainerDied","Data":"a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"}
Jul 15 21:41:38.098486 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:38.098463    2555 scope.go:117] "RemoveContainer" containerID="a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"
Jul 15 21:41:38.099433 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:38.099404    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:38.099528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:38.099471    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:38.099528 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:38.099503    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:39.104944 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:39.104913    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" event={"ID":"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516","Type":"ContainerStarted","Data":"4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca"}
Jul 15 21:41:39.684302 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:39.684261    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:39.749218 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:39.749172    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:39.750737 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:39.750701    2555 patch_prober.go:28] interesting pod/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 21:41:39.750878 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:39.750764    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 21:41:40.749120 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:40.749080    2555 patch_prober.go:28] interesting pod/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 21:41:40.750277 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:40.749794    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 21:41:41.024087 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:41.023975    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:41.749757 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:41.749720    2555 patch_prober.go:28] interesting pod/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 21:41:41.750378 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:41.750345    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 21:41:42.098180 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:42.098107    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:41:42.749792 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:42.749754    2555 patch_prober.go:28] interesting pod/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 15 21:41:42.750415 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:42.750380    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 15 21:41:43.753289 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:43.753252    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:43.754568 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:43.754543    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:41:46.147867 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:46.147792    2555 generic.go:356] "Generic (PLEG): container finished" podID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerID="30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84" exitCode=0
Jul 15 21:41:46.148312 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:46.147866    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" event={"ID":"6db6b571-908f-4553-b36e-c45d56ec70e2","Type":"ContainerDied","Data":"30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"}
Jul 15 21:41:46.148312 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:46.148272    2555 scope.go:117] "RemoveContainer" containerID="30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"
Jul 15 21:41:46.149259 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:46.149229    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:46.149320 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:46.149303    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:46.149357 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:46.149332    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
Jul 15 21:41:47.154999 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:47.154937    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" event={"ID":"6db6b571-908f-4553-b36e-c45d56ec70e2","Type":"ContainerStarted","Data":"90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2"}
Jul 15 21:41:47.639374 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:47.639324    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:47.641082 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:47.640754    2555 patch_prober.go:28] interesting pod/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 15 21:41:47.641082 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:47.640812    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 15 21:41:47.725873 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:47.725796    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:47.826225 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:47.826010    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:48.640034 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:48.640001    2555 patch_prober.go:28] interesting pod/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 15 21:41:48.640609 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:48.640572    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 15 21:41:49.073726 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:49.073641    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:49.640358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:49.640328    2555 patch_prober.go:28] interesting pod/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 15 21:41:49.640828 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:49.640439    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 15 21:41:50.640014 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:50.639976    2555 patch_prober.go:28] interesting pod/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 15 21:41:50.640314 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:50.640280    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 15 21:41:51.643629 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:51.643598    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:51.644703 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:51.644681    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:41:53.141346 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:53.141309    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:55.699419 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:55.699385    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:41:55.799885 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:41:55.799852    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:42:58.088410 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:42:58.088373    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:44:31.902178 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:44:31.902147    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:45:00.143981 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:00.143935    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"]
Jul 15 21:45:00.146269 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:00.146251    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"
Jul 15 21:45:00.148915 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:00.148893    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tl4qg\""
Jul 15 21:45:00.155817 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:00.155796    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"
Jul 15 21:45:00.171396 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:00.171371    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"]
Jul 15 21:45:00.292480 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:00.292453    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"]
Jul 15 21:45:00.295136 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:45:00.295103    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a97723b_1cc5_40bc_8efc_5156d2d676b9.slice/crio-663196d9bf11e7336ce60be915c2e702e55446d20cce6cf956923efc367455d9 WatchSource:0}: Error finding container 663196d9bf11e7336ce60be915c2e702e55446d20cce6cf956923efc367455d9: Status 404 returned error can't find the container with id 663196d9bf11e7336ce60be915c2e702e55446d20cce6cf956923efc367455d9
Jul 15 21:45:01.107514 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:01.107483    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls" event={"ID":"0a97723b-1cc5-40bc-8efc-5156d2d676b9","Type":"ContainerStarted","Data":"a9a448ca39771a7ad51e385aa63a6b1ad54ab29eb0224fc4e5d71dcd16ef4866"}
Jul 15 21:45:01.107696 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:01.107540    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls" event={"ID":"0a97723b-1cc5-40bc-8efc-5156d2d676b9","Type":"ContainerStarted","Data":"663196d9bf11e7336ce60be915c2e702e55446d20cce6cf956923efc367455d9"}
Jul 15 21:45:01.125477 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:01.125417    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls" podStartSLOduration=1.125401283 podStartE2EDuration="1.125401283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:45:01.124808024 +0000 UTC m=+2305.456949135" watchObservedRunningTime="2026-07-15 21:45:01.125401283 +0000 UTC m=+2305.457542373"
Jul 15 21:45:02.114679 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:02.114640    2555 generic.go:356] "Generic (PLEG): container finished" podID="0a97723b-1cc5-40bc-8efc-5156d2d676b9" containerID="a9a448ca39771a7ad51e385aa63a6b1ad54ab29eb0224fc4e5d71dcd16ef4866" exitCode=0
Jul 15 21:45:02.115099 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:02.114692    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls" event={"ID":"0a97723b-1cc5-40bc-8efc-5156d2d676b9","Type":"ContainerDied","Data":"a9a448ca39771a7ad51e385aa63a6b1ad54ab29eb0224fc4e5d71dcd16ef4866"}
Jul 15 21:45:03.256650 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:03.256621    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"
Jul 15 21:45:04.130894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:04.130861    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls" event={"ID":"0a97723b-1cc5-40bc-8efc-5156d2d676b9","Type":"ContainerDied","Data":"663196d9bf11e7336ce60be915c2e702e55446d20cce6cf956923efc367455d9"}
Jul 15 21:45:04.130894 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:04.130880    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"
Jul 15 21:45:04.131146 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:04.130907    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="663196d9bf11e7336ce60be915c2e702e55446d20cce6cf956923efc367455d9"
Jul 15 21:45:05.523423 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:05.523368    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:45:06.473050 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:06.473016    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:45:16.986556 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:16.986505    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:45:44.622347 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:44.622271    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:45:51.970678 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:51.970639    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:45:52.072646 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:45:52.072610    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:22.329268 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:22.329210    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:23.279611 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:23.279568    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:27.386592 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:27.386545    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:36.281311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:36.281286    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:46:36.287172 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:36.287147    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4r92m_a6ee6aff-bea5-4e6f-a88a-fd11d8a4cee4/ovn-acl-logging/0.log"
Jul 15 21:46:36.397621 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:36.397596    2555 scope.go:117] "RemoveContainer" containerID="5186aefa18583d68307002f20a56eee626efcec0458d2f4127d155a43d726ad5"
Jul 15 21:46:48.579145 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:48.579090    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:48.678131 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:48.678092    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:50.811155 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:50.811126    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:56.709235 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:56.709198    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:46:56.820021 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:46:56.819984    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:47:23.389011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:23.388976    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:47:36.633258 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:36.633222    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:47:37.619504 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:37.619442    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:47:42.007152 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.007116    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7447-139900074934704"]
Jul 15 21:47:42.007532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.007491    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a97723b-1cc5-40bc-8efc-5156d2d676b9" containerName="cleanup"
Jul 15 21:47:42.007532 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.007503    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a97723b-1cc5-40bc-8efc-5156d2d676b9" containerName="cleanup"
Jul 15 21:47:42.007604 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.007570    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a97723b-1cc5-40bc-8efc-5156d2d676b9" containerName="cleanup"
Jul 15 21:47:42.011135 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.011115    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:42.014490 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.014470    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ncn7w\""
Jul 15 21:47:42.016407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.016380    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139900074934704"]
Jul 15 21:47:42.037466 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.037441    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdtl\" (UniqueName: \"kubernetes.io/projected/c5cec59a-bc2b-49ce-8052-972103afe41d-kube-api-access-nhdtl\") pod \"test-curl-7447-139900074934704\" (UID: \"c5cec59a-bc2b-49ce-8052-972103afe41d\") " pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:42.138343 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.138306    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdtl\" (UniqueName: \"kubernetes.io/projected/c5cec59a-bc2b-49ce-8052-972103afe41d-kube-api-access-nhdtl\") pod \"test-curl-7447-139900074934704\" (UID: \"c5cec59a-bc2b-49ce-8052-972103afe41d\") " pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:42.148574 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.148545    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdtl\" (UniqueName: \"kubernetes.io/projected/c5cec59a-bc2b-49ce-8052-972103afe41d-kube-api-access-nhdtl\") pod \"test-curl-7447-139900074934704\" (UID: \"c5cec59a-bc2b-49ce-8052-972103afe41d\") " pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:42.324048 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.323940    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:42.454456 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.454434    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139900074934704"]
Jul 15 21:47:42.456499 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:47:42.456452    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5cec59a_bc2b_49ce_8052_972103afe41d.slice/crio-0c18805ffa3f0d89624df1dfc9af38ea55ff93f5b51d0d87ff148fd83b61533d WatchSource:0}: Error finding container 0c18805ffa3f0d89624df1dfc9af38ea55ff93f5b51d0d87ff148fd83b61533d: Status 404 returned error can't find the container with id 0c18805ffa3f0d89624df1dfc9af38ea55ff93f5b51d0d87ff148fd83b61533d
Jul 15 21:47:42.457776 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.457757    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:47:42.928890 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:42.928856    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139900074934704" event={"ID":"c5cec59a-bc2b-49ce-8052-972103afe41d","Type":"ContainerStarted","Data":"0c18805ffa3f0d89624df1dfc9af38ea55ff93f5b51d0d87ff148fd83b61533d"}
Jul 15 21:47:43.935785 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:43.935748    2555 generic.go:356] "Generic (PLEG): container finished" podID="c5cec59a-bc2b-49ce-8052-972103afe41d" containerID="9c3f10aab9b25e9f64a5d0fb8b3901a3774b540561c6394443d0cb9901d6b08d" exitCode=0
Jul 15 21:47:43.936184 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:43.935829    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139900074934704" event={"ID":"c5cec59a-bc2b-49ce-8052-972103afe41d","Type":"ContainerDied","Data":"9c3f10aab9b25e9f64a5d0fb8b3901a3774b540561c6394443d0cb9901d6b08d"}
Jul 15 21:47:45.073396 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.073376    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:45.097540 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.097506    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7447-139900074934704_c5cec59a-bc2b-49ce-8052-972103afe41d/test-curl-7447-139900074934704/0.log"
Jul 15 21:47:45.126148 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.126118    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7447-139900074934704"]
Jul 15 21:47:45.129046 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.129022    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7447-139900074934704"]
Jul 15 21:47:45.159189 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.159164    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhdtl\" (UniqueName: \"kubernetes.io/projected/c5cec59a-bc2b-49ce-8052-972103afe41d-kube-api-access-nhdtl\") pod \"c5cec59a-bc2b-49ce-8052-972103afe41d\" (UID: \"c5cec59a-bc2b-49ce-8052-972103afe41d\") "
Jul 15 21:47:45.161323 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.161295    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5cec59a-bc2b-49ce-8052-972103afe41d-kube-api-access-nhdtl" (OuterVolumeSpecName: "kube-api-access-nhdtl") pod "c5cec59a-bc2b-49ce-8052-972103afe41d" (UID: "c5cec59a-bc2b-49ce-8052-972103afe41d"). InnerVolumeSpecName "kube-api-access-nhdtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:47:45.260678 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.260571    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhdtl\" (UniqueName: \"kubernetes.io/projected/c5cec59a-bc2b-49ce-8052-972103afe41d-kube-api-access-nhdtl\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:47:45.292973 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.292922    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7447-139900075079408"]
Jul 15 21:47:45.293316 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.293303    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5cec59a-bc2b-49ce-8052-972103afe41d" containerName="test-curl-7447-139900074934704"
Jul 15 21:47:45.293360 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.293318    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5cec59a-bc2b-49ce-8052-972103afe41d" containerName="test-curl-7447-139900074934704"
Jul 15 21:47:45.293425 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.293415    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5cec59a-bc2b-49ce-8052-972103afe41d" containerName="test-curl-7447-139900074934704"
Jul 15 21:47:45.296377 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.296360    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:45.304699 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.304675    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139900075079408"]
Jul 15 21:47:45.361816 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.361784    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x284k\" (UniqueName: \"kubernetes.io/projected/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd-kube-api-access-x284k\") pod \"test-curl-7447-139900075079408\" (UID: \"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd\") " pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:45.463172 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.463136    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x284k\" (UniqueName: \"kubernetes.io/projected/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd-kube-api-access-x284k\") pod \"test-curl-7447-139900075079408\" (UID: \"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd\") " pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:45.472651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.472624    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x284k\" (UniqueName: \"kubernetes.io/projected/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd-kube-api-access-x284k\") pod \"test-curl-7447-139900075079408\" (UID: \"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd\") " pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:45.606183 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.606098    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:45.733787 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.733760    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139900075079408"]
Jul 15 21:47:45.735404 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:47:45.735375    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0c1ce48_0c9d_4047_a654_c0e5e13a85cd.slice/crio-dcf19cef0b2b3a9e84f805cd6beb6c88f57ea8de8f15dec8cb7063e41005870e WatchSource:0}: Error finding container dcf19cef0b2b3a9e84f805cd6beb6c88f57ea8de8f15dec8cb7063e41005870e: Status 404 returned error can't find the container with id dcf19cef0b2b3a9e84f805cd6beb6c88f57ea8de8f15dec8cb7063e41005870e
Jul 15 21:47:45.947150 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.947115    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139900075079408" event={"ID":"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd","Type":"ContainerStarted","Data":"dcf19cef0b2b3a9e84f805cd6beb6c88f57ea8de8f15dec8cb7063e41005870e"}
Jul 15 21:47:45.948371 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.948352    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:45.948501 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:45.948356    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c18805ffa3f0d89624df1dfc9af38ea55ff93f5b51d0d87ff148fd83b61533d"
Jul 15 21:47:45.959403 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:47:45.959376    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7447-139900074934704\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c5cec59a-bc2b-49ce-8052-972103afe41d" pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:46.215927 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:47:46.215845    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7447-139900074934704\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="c5cec59a-bc2b-49ce-8052-972103afe41d" pod="opendatahub/test-curl-7447-139900074934704"
Jul 15 21:47:46.217510 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:46.217484    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5cec59a-bc2b-49ce-8052-972103afe41d" path="/var/lib/kubelet/pods/c5cec59a-bc2b-49ce-8052-972103afe41d/volumes"
Jul 15 21:47:46.955319 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:46.955284    2555 generic.go:356] "Generic (PLEG): container finished" podID="c0c1ce48-0c9d-4047-a654-c0e5e13a85cd" containerID="0fe1a3e863f02775ed1d7747bdceb1bd497fca5713b88099b56efd86ca809559" exitCode=0
Jul 15 21:47:46.955476 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:46.955359    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139900075079408" event={"ID":"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd","Type":"ContainerDied","Data":"0fe1a3e863f02775ed1d7747bdceb1bd497fca5713b88099b56efd86ca809559"}
Jul 15 21:47:48.092738 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.092714    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:48.117623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.117588    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7447-139900075079408_c0c1ce48-0c9d-4047-a654-c0e5e13a85cd/test-curl-7447-139900075079408/0.log"
Jul 15 21:47:48.145358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.145329    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7447-139900075079408"]
Jul 15 21:47:48.152407 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.152377    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7447-139900075079408"]
Jul 15 21:47:48.188832 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.188805    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x284k\" (UniqueName: \"kubernetes.io/projected/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd-kube-api-access-x284k\") pod \"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd\" (UID: \"c0c1ce48-0c9d-4047-a654-c0e5e13a85cd\") "
Jul 15 21:47:48.190773 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.190749    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd-kube-api-access-x284k" (OuterVolumeSpecName: "kube-api-access-x284k") pod "c0c1ce48-0c9d-4047-a654-c0e5e13a85cd" (UID: "c0c1ce48-0c9d-4047-a654-c0e5e13a85cd"). InnerVolumeSpecName "kube-api-access-x284k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:47:48.225903 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.218850    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0c1ce48-0c9d-4047-a654-c0e5e13a85cd" path="/var/lib/kubelet/pods/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd/volumes"
Jul 15 21:47:48.290204 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.290173    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x284k\" (UniqueName: \"kubernetes.io/projected/c0c1ce48-0c9d-4047-a654-c0e5e13a85cd-kube-api-access-x284k\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:47:48.507712 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.507632    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:47:48.966104 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.966077    2555 scope.go:117] "RemoveContainer" containerID="0fe1a3e863f02775ed1d7747bdceb1bd497fca5713b88099b56efd86ca809559"
Jul 15 21:47:48.966283 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:48.966081    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075079408"
Jul 15 21:47:51.024000 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.023964    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7447-139900075080416"]
Jul 15 21:47:51.024487 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.024352    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c1ce48-0c9d-4047-a654-c0e5e13a85cd" containerName="test-curl-7447-139900075079408"
Jul 15 21:47:51.024487 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.024364    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c1ce48-0c9d-4047-a654-c0e5e13a85cd" containerName="test-curl-7447-139900075079408"
Jul 15 21:47:51.024487 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.024432    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c1ce48-0c9d-4047-a654-c0e5e13a85cd" containerName="test-curl-7447-139900075079408"
Jul 15 21:47:51.028701 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.028683    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:51.031423 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.031401    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ncn7w\""
Jul 15 21:47:51.035345 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.035315    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139900075080416"]
Jul 15 21:47:51.115623 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.115593    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrkgp\" (UniqueName: \"kubernetes.io/projected/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e-kube-api-access-lrkgp\") pod \"test-curl-7447-139900075080416\" (UID: \"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e\") " pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:51.216595 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.216563    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrkgp\" (UniqueName: \"kubernetes.io/projected/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e-kube-api-access-lrkgp\") pod \"test-curl-7447-139900075080416\" (UID: \"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e\") " pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:51.228210 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.228173    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrkgp\" (UniqueName: \"kubernetes.io/projected/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e-kube-api-access-lrkgp\") pod \"test-curl-7447-139900075080416\" (UID: \"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e\") " pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:51.340086 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.339998    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:51.472800 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.472775    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139900075080416"]
Jul 15 21:47:51.475344 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:47:51.475310    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02bd37b5_0e8f_42c0_ab1a_f8949a663d8e.slice/crio-22066459d9a2f64bdc5ea0bc3a69e58327717c064ae1cec4e1eb5b9c0c597782 WatchSource:0}: Error finding container 22066459d9a2f64bdc5ea0bc3a69e58327717c064ae1cec4e1eb5b9c0c597782: Status 404 returned error can't find the container with id 22066459d9a2f64bdc5ea0bc3a69e58327717c064ae1cec4e1eb5b9c0c597782
Jul 15 21:47:51.984227 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.984193    2555 generic.go:356] "Generic (PLEG): container finished" podID="02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" containerID="595f2ad8a6e8487cc58487e529faa4df13dd8f8e887c54ea359ddf63ef5c1686" exitCode=0
Jul 15 21:47:51.984382 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.984278    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139900075080416" event={"ID":"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e","Type":"ContainerDied","Data":"595f2ad8a6e8487cc58487e529faa4df13dd8f8e887c54ea359ddf63ef5c1686"}
Jul 15 21:47:51.984382 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:51.984317    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139900075080416" event={"ID":"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e","Type":"ContainerStarted","Data":"22066459d9a2f64bdc5ea0bc3a69e58327717c064ae1cec4e1eb5b9c0c597782"}
Jul 15 21:47:53.122924 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.122901    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:53.148135 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.148106    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7447-139900075080416_02bd37b5-0e8f-42c0-ab1a-f8949a663d8e/test-curl-7447-139900075080416/0.log"
Jul 15 21:47:53.185739 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.185710    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7447-139900075080416"]
Jul 15 21:47:53.190001 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.189938    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7447-139900075080416"]
Jul 15 21:47:53.234970 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.234930    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrkgp\" (UniqueName: \"kubernetes.io/projected/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e-kube-api-access-lrkgp\") pod \"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e\" (UID: \"02bd37b5-0e8f-42c0-ab1a-f8949a663d8e\") "
Jul 15 21:47:53.237024 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.237001    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e-kube-api-access-lrkgp" (OuterVolumeSpecName: "kube-api-access-lrkgp") pod "02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" (UID: "02bd37b5-0e8f-42c0-ab1a-f8949a663d8e"). InnerVolumeSpecName "kube-api-access-lrkgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:47:53.336196 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.336161    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrkgp\" (UniqueName: \"kubernetes.io/projected/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e-kube-api-access-lrkgp\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:47:53.995479 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.995447    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:53.995479 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:53.995471    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22066459d9a2f64bdc5ea0bc3a69e58327717c064ae1cec4e1eb5b9c0c597782"
Jul 15 21:47:54.005283 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:47:54.005252    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7447-139900075080416\" is forbidden: User \"system:node:ip-10-0-128-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-44.ec2.internal' and this object" podUID="02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" pod="opendatahub/test-curl-7447-139900075080416"
Jul 15 21:47:54.217295 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:54.217264    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" path="/var/lib/kubelet/pods/02bd37b5-0e8f-42c0-ab1a-f8949a663d8e/volumes"
Jul 15 21:47:56.466931 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:47:56.466901    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:02.501896 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.501859    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"]
Jul 15 21:48:02.502563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.502182    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" podUID="dee527f2-0282-452e-a713-7fb8c51cbaa8" containerName="maas-api" containerID="cri-o://becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d" gracePeriod=30
Jul 15 21:48:02.531355 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.531330    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:02.755938 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.755874    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:48:02.820565 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.820528    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzkc2\" (UniqueName: \"kubernetes.io/projected/dee527f2-0282-452e-a713-7fb8c51cbaa8-kube-api-access-wzkc2\") pod \"dee527f2-0282-452e-a713-7fb8c51cbaa8\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") "
Jul 15 21:48:02.820759 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.820619    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee527f2-0282-452e-a713-7fb8c51cbaa8-openshift-service-ca\") pod \"dee527f2-0282-452e-a713-7fb8c51cbaa8\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") "
Jul 15 21:48:02.820759 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.820666    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls\") pod \"dee527f2-0282-452e-a713-7fb8c51cbaa8\" (UID: \"dee527f2-0282-452e-a713-7fb8c51cbaa8\") "
Jul 15 21:48:02.821084 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.821047    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee527f2-0282-452e-a713-7fb8c51cbaa8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dee527f2-0282-452e-a713-7fb8c51cbaa8" (UID: "dee527f2-0282-452e-a713-7fb8c51cbaa8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:48:02.822729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.822701    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dee527f2-0282-452e-a713-7fb8c51cbaa8" (UID: "dee527f2-0282-452e-a713-7fb8c51cbaa8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:48:02.822811 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.822732    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee527f2-0282-452e-a713-7fb8c51cbaa8-kube-api-access-wzkc2" (OuterVolumeSpecName: "kube-api-access-wzkc2") pod "dee527f2-0282-452e-a713-7fb8c51cbaa8" (UID: "dee527f2-0282-452e-a713-7fb8c51cbaa8"). InnerVolumeSpecName "kube-api-access-wzkc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:48:02.922107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.922074    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzkc2\" (UniqueName: \"kubernetes.io/projected/dee527f2-0282-452e-a713-7fb8c51cbaa8-kube-api-access-wzkc2\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:02.922107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.922100    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee527f2-0282-452e-a713-7fb8c51cbaa8-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:02.922107 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:02.922110    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dee527f2-0282-452e-a713-7fb8c51cbaa8-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:03.044651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.044562    2555 generic.go:356] "Generic (PLEG): container finished" podID="dee527f2-0282-452e-a713-7fb8c51cbaa8" containerID="becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d" exitCode=0
Jul 15 21:48:03.044651 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.044610    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"
Jul 15 21:48:03.044877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.044653    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" event={"ID":"dee527f2-0282-452e-a713-7fb8c51cbaa8","Type":"ContainerDied","Data":"becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d"}
Jul 15 21:48:03.044877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.044703    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf" event={"ID":"dee527f2-0282-452e-a713-7fb8c51cbaa8","Type":"ContainerDied","Data":"a50ee3352c5f816438f44c6af8972a4c068fa2030fb1044a1cd78aa0ed1d0081"}
Jul 15 21:48:03.044877 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.044727    2555 scope.go:117] "RemoveContainer" containerID="becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d"
Jul 15 21:48:03.054874 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.054854    2555 scope.go:117] "RemoveContainer" containerID="becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d"
Jul 15 21:48:03.055232 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:48:03.055213    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d\": container with ID starting with becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d not found: ID does not exist" containerID="becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d"
Jul 15 21:48:03.055311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.055239    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d"} err="failed to get container status \"becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d\": rpc error: code = NotFound desc = could not find container \"becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d\": container with ID starting with becab4cf4704b345944861b8559ff9ffc63928214ccd804ae9dd22489752a37d not found: ID does not exist"
Jul 15 21:48:03.063051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.063025    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:03.067687 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.067666    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"]
Jul 15 21:48:03.073297 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:03.073275    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fa020c-689679759c-hnjsf"]
Jul 15 21:48:04.217449 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:04.217413    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dee527f2-0282-452e-a713-7fb8c51cbaa8" path="/var/lib/kubelet/pods/dee527f2-0282-452e-a713-7fb8c51cbaa8/volumes"
Jul 15 21:48:09.047927 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:09.047893    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:09.148530 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:09.148491    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:09.324662 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:09.324576    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"]
Jul 15 21:48:09.324884 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:09.324850    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy" containerID="cri-o://4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca" gracePeriod=30
Jul 15 21:48:09.338372 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:09.338347    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:09.569127 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:09.569096    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:10.056160 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:10.056130    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:13.756453 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:13.756414    2555 patch_prober.go:28] interesting pod/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 21:48:13.756820 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:13.756474    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 21:48:14.575822 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.575792    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:48:14.727338 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727304    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-socket\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727353    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-data\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727374    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-credential-socket\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727415    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-certs\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727432    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-token\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727451    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdv4z\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-kube-api-access-bdv4z\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727476    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-envoy\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727537 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727536    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istiod-ca-cert\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.727914 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727596    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-podinfo\") pod \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\" (UID: \"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516\") "
Jul 15 21:48:14.728029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727651    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:14.728165 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.728029    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:48:14.728165 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727739    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:14.728165 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.727910    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:14.728512 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.728489    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:14.729833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.729809    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:14.729925 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.729854    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:48:14.730129 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.730106    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-kube-api-access-bdv4z" (OuterVolumeSpecName: "kube-api-access-bdv4z") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "kube-api-access-bdv4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:48:14.730192 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.730139    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" (UID: "5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:48:14.828477 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828450    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828477 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828476    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828486    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828495    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828503    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdv4z\" (UniqueName: \"kubernetes.io/projected/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-kube-api-access-bdv4z\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828511    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828519    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828527    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:14.828833 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:14.828534    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:15.105434 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.105341    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerID="4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca" exitCode=0
Jul 15 21:48:15.105583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.105431    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" event={"ID":"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516","Type":"ContainerDied","Data":"4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca"}
Jul 15 21:48:15.105583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.105442    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"
Jul 15 21:48:15.105583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.105482    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv" event={"ID":"5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516","Type":"ContainerDied","Data":"d7910394f398d4c304d5f3fbcec605ef01ad20aa53cc239cc31952a626992c5f"}
Jul 15 21:48:15.105583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.105505    2555 scope.go:117] "RemoveContainer" containerID="4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca"
Jul 15 21:48:15.114570 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.114555    2555 scope.go:117] "RemoveContainer" containerID="a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"
Jul 15 21:48:15.123181 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.123158    2555 scope.go:117] "RemoveContainer" containerID="4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca"
Jul 15 21:48:15.123446 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:48:15.123421    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca\": container with ID starting with 4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca not found: ID does not exist" containerID="4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca"
Jul 15 21:48:15.123508 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.123460    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca"} err="failed to get container status \"4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca\": rpc error: code = NotFound desc = could not find container \"4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca\": container with ID starting with 4ecea5b4100256ce06374e836983cf2f88d02276c69519edfd08ab0019dd76ca not found: ID does not exist"
Jul 15 21:48:15.123508 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.123487    2555 scope.go:117] "RemoveContainer" containerID="a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"
Jul 15 21:48:15.123742 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:48:15.123724    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead\": container with ID starting with a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead not found: ID does not exist" containerID="a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"
Jul 15 21:48:15.123792 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.123747    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead"} err="failed to get container status \"a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead\": rpc error: code = NotFound desc = could not find container \"a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead\": container with ID starting with a6b0b719a0f624d3f873cabaa1bd23afbe1b94dd04dc3fedef3a1e81321baead not found: ID does not exist"
Jul 15 21:48:15.128645 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.128627    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"]
Jul 15 21:48:15.133606 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.133587    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-fa020c-openshift-default-7445866479-h7jkv"]
Jul 15 21:48:15.891719 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.891685    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"]
Jul 15 21:48:15.892330 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.891943    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw" podUID="9f86d951-eba7-4b69-abfa-fc2469fa827f" containerName="maas-api" containerID="cri-o://7fe8a366e1e36e21c74423966c05370ff9dc0260847547853876a03d9e0fbcdf" gracePeriod=30
Jul 15 21:48:15.919741 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.919710    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"]
Jul 15 21:48:15.922160 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.922137    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-7ef33e-29735865-s4jls"]
Jul 15 21:48:15.923101 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:15.923079    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:16.112729 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.112697    2555 generic.go:356] "Generic (PLEG): container finished" podID="9f86d951-eba7-4b69-abfa-fc2469fa827f" containerID="7fe8a366e1e36e21c74423966c05370ff9dc0260847547853876a03d9e0fbcdf" exitCode=0
Jul 15 21:48:16.112907 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.112773    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw" event={"ID":"9f86d951-eba7-4b69-abfa-fc2469fa827f","Type":"ContainerDied","Data":"7fe8a366e1e36e21c74423966c05370ff9dc0260847547853876a03d9e0fbcdf"}
Jul 15 21:48:16.146202 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.146142    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:48:16.217392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.217353    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a97723b-1cc5-40bc-8efc-5156d2d676b9" path="/var/lib/kubelet/pods/0a97723b-1cc5-40bc-8efc-5156d2d676b9/volumes"
Jul 15 21:48:16.217989 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.217943    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" path="/var/lib/kubelet/pods/5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516/volumes"
Jul 15 21:48:16.239869 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.239845    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f86d951-eba7-4b69-abfa-fc2469fa827f-maas-api-tls\") pod \"9f86d951-eba7-4b69-abfa-fc2469fa827f\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") "
Jul 15 21:48:16.240022 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.239881    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f86d951-eba7-4b69-abfa-fc2469fa827f-openshift-service-ca\") pod \"9f86d951-eba7-4b69-abfa-fc2469fa827f\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") "
Jul 15 21:48:16.240022 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.239908    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr8cx\" (UniqueName: \"kubernetes.io/projected/9f86d951-eba7-4b69-abfa-fc2469fa827f-kube-api-access-xr8cx\") pod \"9f86d951-eba7-4b69-abfa-fc2469fa827f\" (UID: \"9f86d951-eba7-4b69-abfa-fc2469fa827f\") "
Jul 15 21:48:16.240256 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.240237    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f86d951-eba7-4b69-abfa-fc2469fa827f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f86d951-eba7-4b69-abfa-fc2469fa827f" (UID: "9f86d951-eba7-4b69-abfa-fc2469fa827f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:48:16.242056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.242029    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f86d951-eba7-4b69-abfa-fc2469fa827f-kube-api-access-xr8cx" (OuterVolumeSpecName: "kube-api-access-xr8cx") pod "9f86d951-eba7-4b69-abfa-fc2469fa827f" (UID: "9f86d951-eba7-4b69-abfa-fc2469fa827f"). InnerVolumeSpecName "kube-api-access-xr8cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:48:16.242127 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.242076    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f86d951-eba7-4b69-abfa-fc2469fa827f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f86d951-eba7-4b69-abfa-fc2469fa827f" (UID: "9f86d951-eba7-4b69-abfa-fc2469fa827f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:48:16.341406 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.341371    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f86d951-eba7-4b69-abfa-fc2469fa827f-maas-api-tls\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:16.341406 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.341399    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f86d951-eba7-4b69-abfa-fc2469fa827f-openshift-service-ca\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:16.341406 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.341410    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr8cx\" (UniqueName: \"kubernetes.io/projected/9f86d951-eba7-4b69-abfa-fc2469fa827f-kube-api-access-xr8cx\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:16.625604 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:16.625561    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:17.120916 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:17.120838    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw" event={"ID":"9f86d951-eba7-4b69-abfa-fc2469fa827f","Type":"ContainerDied","Data":"7f0c1c2449e628daa40a24eb3efb5b69576cfaf8755cf4f41208da39f6a1c826"}
Jul 15 21:48:17.120916 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:17.120899    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"
Jul 15 21:48:17.121379 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:17.120918    2555 scope.go:117] "RemoveContainer" containerID="7fe8a366e1e36e21c74423966c05370ff9dc0260847547853876a03d9e0fbcdf"
Jul 15 21:48:17.147164 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:17.147136    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"]
Jul 15 21:48:17.151978 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:17.151945    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7ef33e-59d98c4d99-rh6jw"]
Jul 15 21:48:18.217313 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:18.217278    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f86d951-eba7-4b69-abfa-fc2469fa827f" path="/var/lib/kubelet/pods/9f86d951-eba7-4b69-abfa-fc2469fa827f/volumes"
Jul 15 21:48:22.847504 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:22.847465    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:22.904635 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:22.904611    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:23.097916 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:23.097832    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"]
Jul 15 21:48:23.098176 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:23.098149    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy" containerID="cri-o://90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2" gracePeriod=30
Jul 15 21:48:23.124306 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:23.124270    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:23.352408 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:23.352312    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:28.372583 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.372528    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:48:28.548250 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548213    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db6b571-908f-4553-b36e-c45d56ec70e2-istiod-ca-cert\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548257    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5gjh\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-kube-api-access-n5gjh\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548293    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-socket\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548329    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-podinfo\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548348    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-envoy\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548401 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548373    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-data\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548518    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-certs\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548557    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-credential-socket\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548617    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-token\") pod \"6db6b571-908f-4553-b36e-c45d56ec70e2\" (UID: \"6db6b571-908f-4553-b36e-c45d56ec70e2\") "
Jul 15 21:48:28.548665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548629    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:28.548665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548634    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6db6b571-908f-4553-b36e-c45d56ec70e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:48:28.548934 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548836    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:28.548934 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548862    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:28.548934 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548923    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db6b571-908f-4553-b36e-c45d56ec70e2-istiod-ca-cert\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.549140 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548941    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.549140 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548979    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-workload-certs\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.549140 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.548992    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-credential-socket\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.549348 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.549323    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:28.550617 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.550587    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-kube-api-access-n5gjh" (OuterVolumeSpecName: "kube-api-access-n5gjh") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "kube-api-access-n5gjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:48:28.550724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.550613    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:48:28.550724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.550653    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 21:48:28.550724 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.550718    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "6db6b571-908f-4553-b36e-c45d56ec70e2" (UID: "6db6b571-908f-4553-b36e-c45d56ec70e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:48:28.649738 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.649708    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-token\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.649738 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.649737    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5gjh\" (UniqueName: \"kubernetes.io/projected/6db6b571-908f-4553-b36e-c45d56ec70e2-kube-api-access-n5gjh\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.650029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.649747    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-podinfo\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.650029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.649757    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-envoy\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:28.650029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:28.649766    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db6b571-908f-4553-b36e-c45d56ec70e2-istio-data\") on node \"ip-10-0-128-44.ec2.internal\" DevicePath \"\""
Jul 15 21:48:29.186218 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.186190    2555 generic.go:356] "Generic (PLEG): container finished" podID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerID="90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2" exitCode=0
Jul 15 21:48:29.186392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.186235    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"
Jul 15 21:48:29.186392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.186276    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" event={"ID":"6db6b571-908f-4553-b36e-c45d56ec70e2","Type":"ContainerDied","Data":"90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2"}
Jul 15 21:48:29.186392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.186327    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz" event={"ID":"6db6b571-908f-4553-b36e-c45d56ec70e2","Type":"ContainerDied","Data":"0f6f585fa564ea0b35b09ce188972ca48f92a1fca41ea7c394d6a83d913cd1c7"}
Jul 15 21:48:29.186392 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.186349    2555 scope.go:117] "RemoveContainer" containerID="90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2"
Jul 15 21:48:29.195707 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.195690    2555 scope.go:117] "RemoveContainer" containerID="30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"
Jul 15 21:48:29.204455 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.204434    2555 scope.go:117] "RemoveContainer" containerID="90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2"
Jul 15 21:48:29.204759 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:48:29.204741    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2\": container with ID starting with 90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2 not found: ID does not exist" containerID="90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2"
Jul 15 21:48:29.204801 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.204768    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2"} err="failed to get container status \"90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2\": rpc error: code = NotFound desc = could not find container \"90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2\": container with ID starting with 90edcacc83caede2d7253dd6da513a8e807d5c0866ca4d3e340015506fa1bca2 not found: ID does not exist"
Jul 15 21:48:29.204801 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.204784    2555 scope.go:117] "RemoveContainer" containerID="30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"
Jul 15 21:48:29.205043 ip-10-0-128-44 kubenswrapper[2555]: E0715 21:48:29.205025    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84\": container with ID starting with 30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84 not found: ID does not exist" containerID="30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"
Jul 15 21:48:29.205114 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.205048    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84"} err="failed to get container status \"30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84\": rpc error: code = NotFound desc = could not find container \"30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84\": container with ID starting with 30b0a9994321e6ac729089306dfa4e4557709905790842ef356664b0255b8b84 not found: ID does not exist"
Jul 15 21:48:29.216624 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.216588    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"]
Jul 15 21:48:29.218091 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.218065    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-7ef33e-openshift-default-7855c659f5-64sfz"]
Jul 15 21:48:29.340311 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.340278    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6pscc_b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89/manager/1.log"
Jul 15 21:48:29.466931 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.466844    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5s7kk_550cae76-7a34-4e8a-bad1-caca322ec031/manager/1.log"
Jul 15 21:48:29.738178 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.738088    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-prn5d_9564402b-7e1e-4275-aee5-adbe06c3c21b/server/0.log"
Jul 15 21:48:29.856161 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:29.856123    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j5z24_e87f8b62-a84a-479d-b8ea-44ec05e51993/manager/1.log"
Jul 15 21:48:30.216728 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:30.216695    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" path="/var/lib/kubelet/pods/6db6b571-908f-4553-b36e-c45d56ec70e2/volumes"
Jul 15 21:48:30.220995 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:30.220977    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-858d4f578f-rtk74_42978d06-f6bc-4e7b-9ca4-83606ade276f/copy-manifests/0.log"
Jul 15 21:48:30.229051 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:30.229031    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-858d4f578f-rtk74_42978d06-f6bc-4e7b-9ca4-83606ade276f/manager/1.log"
Jul 15 21:48:30.594895 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:30.594824    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-75f458d465-zc65p_ed662043-f1b8-4c43-9013-37d73ef9c91d/maas-api/0.log"
Jul 15 21:48:30.727358 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:30.727328    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735850-txbpp_7ffa91ca-5765-41d5-9f07-39779530a32c/cleanup/0.log"
Jul 15 21:48:30.952385 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:30.952354    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ff544_53a2f994-c067-42b1-8fff-8c9b3089881e/postgres/0.log"
Jul 15 21:48:32.392372 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:32.392329    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r9qr5_4295ecb3-f335-4757-8d89-ee38705182d3/manager/1.log"
Jul 15 21:48:32.505225 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:32.505198    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m5bp4_dd4a1644-4a84-4e34-bdbc-17b46a830d25/manager/1.log"
Jul 15 21:48:32.617860 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:32.617837    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2g7rk_e1601410-2ba4-4dbe-9a3c-220b25bc0d05/kuadrant-console-plugin/0.log"
Jul 15 21:48:32.736147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:32.736071    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vtp2s_80fcc130-f349-4cf8-81bc-2d11a079fae0/registry-server/0.log"
Jul 15 21:48:32.857667 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:32.857633    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vnzjp_47d03303-830c-42de-93f8-d8348cead365/manager/0.log"
Jul 15 21:48:33.108639 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:33.108568    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tqm4k_45cfec9f-bc04-497a-bc58-388bd4379e9b/manager/1.log"
Jul 15 21:48:33.830508 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:33.830454    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bcb5978cd-w5cpd_d05f479c-075c-4e14-902f-6c3571e0418d/kube-auth-proxy/0.log"
Jul 15 21:48:34.190390 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:34.190342    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-svmk4_b9f5ec2c-610b-4946-98d4-429987b4a29d/payload-processing/0.log"
Jul 15 21:48:34.891242 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:34.891212    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lszcm_5207ad27-b704-4b7f-8caa-2dfeeee1ae73/main/0.log"
Jul 15 21:48:40.923652 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:40.923603    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:41.023296 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:41.023255    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log" limit=1024
Jul 15 21:48:43.978080 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:43.978051    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zccbq_543fcfc2-8dba-49bd-a5f4-5a9064532860/global-pull-secret-syncer/0.log"
Jul 15 21:48:44.041069 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:44.041043    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mnpcr_ab1f5324-3152-4e5c-b9d3-57007a007d8c/konnectivity-agent/0.log"
Jul 15 21:48:44.133739 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:44.133711    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-44.ec2.internal_72a16374957430e777a4eeb37d6601d6/haproxy/0.log"
Jul 15 21:48:48.955664 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:48.955630    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r9qr5_4295ecb3-f335-4757-8d89-ee38705182d3/manager/1.log"
Jul 15 21:48:48.957709 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:48.957692    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r9qr5_4295ecb3-f335-4757-8d89-ee38705182d3/manager/0.log"
Jul 15 21:48:48.987521 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:48.987497    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m5bp4_dd4a1644-4a84-4e34-bdbc-17b46a830d25/manager/1.log"
Jul 15 21:48:48.987627 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:48.987598    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m5bp4_dd4a1644-4a84-4e34-bdbc-17b46a830d25/manager/0.log"
Jul 15 21:48:49.012301 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:49.012278    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2g7rk_e1601410-2ba4-4dbe-9a3c-220b25bc0d05/kuadrant-console-plugin/0.log"
Jul 15 21:48:49.060348 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:49.060321    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vtp2s_80fcc130-f349-4cf8-81bc-2d11a079fae0/registry-server/0.log"
Jul 15 21:48:49.176473 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:49.176440    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vnzjp_47d03303-830c-42de-93f8-d8348cead365/manager/0.log"
Jul 15 21:48:49.260011 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:49.259891    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tqm4k_45cfec9f-bc04-497a-bc58-388bd4379e9b/manager/0.log"
Jul 15 21:48:49.272705 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:49.272682    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tqm4k_45cfec9f-bc04-497a-bc58-388bd4379e9b/manager/1.log"
Jul 15 21:48:51.545641 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.545612    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5rbbn_c3dbfa95-7bcf-42ea-a451-acd94369b0e7/node-exporter/0.log"
Jul 15 21:48:51.575686 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.575661    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5rbbn_c3dbfa95-7bcf-42ea-a451-acd94369b0e7/kube-rbac-proxy/0.log"
Jul 15 21:48:51.605665 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.605635    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5rbbn_c3dbfa95-7bcf-42ea-a451-acd94369b0e7/init-textfile/0.log"
Jul 15 21:48:51.851468 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.851441    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/prometheus/0.log"
Jul 15 21:48:51.879635 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.879613    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/config-reloader/0.log"
Jul 15 21:48:51.906931 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.906911    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/thanos-sidecar/0.log"
Jul 15 21:48:51.937029 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.937008    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/kube-rbac-proxy-web/0.log"
Jul 15 21:48:51.964333 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.964310    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/kube-rbac-proxy/0.log"
Jul 15 21:48:51.991445 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:51.991388    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/kube-rbac-proxy-thanos/0.log"
Jul 15 21:48:52.019801 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:52.019776    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3c8ec6a8-4307-4f88-a405-4f0a10345e2b/init-config-reloader/0.log"
Jul 15 21:48:54.122270 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122235    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"]
Jul 15 21:48:54.122644 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122617    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy"
Jul 15 21:48:54.122644 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122628    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy"
Jul 15 21:48:54.122644 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122638    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy"
Jul 15 21:48:54.122644 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122644    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122652    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122658    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122672    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dee527f2-0282-452e-a713-7fb8c51cbaa8" containerName="maas-api"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122678    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dee527f2-0282-452e-a713-7fb8c51cbaa8" containerName="maas-api"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122683    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" containerName="test-curl-7447-139900075080416"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122689    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" containerName="test-curl-7447-139900075080416"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122697    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122704    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122715    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f86d951-eba7-4b69-abfa-fc2469fa827f" containerName="maas-api"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122720    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f86d951-eba7-4b69-abfa-fc2469fa827f" containerName="maas-api"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122771    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dee527f2-0282-452e-a713-7fb8c51cbaa8" containerName="maas-api"
Jul 15 21:48:54.122774 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122778    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bd37b5-0e8f-42c0-ab1a-f8949a663d8e" containerName="test-curl-7447-139900075080416"
Jul 15 21:48:54.123157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122786    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f86d951-eba7-4b69-abfa-fc2469fa827f" containerName="maas-api"
Jul 15 21:48:54.123157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122792    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6db6b571-908f-4553-b36e-c45d56ec70e2" containerName="istio-proxy"
Jul 15 21:48:54.123157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122798    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy"
Jul 15 21:48:54.123157 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.122804    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bc0ab2d-4fe8-4608-9d82-fcfdfe87f516" containerName="istio-proxy"
Jul 15 21:48:54.125525 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.125507    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.128262 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.128237    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-546bh\"/\"openshift-service-ca.crt\""
Jul 15 21:48:54.128395 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.128312    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-546bh\"/\"kube-root-ca.crt\""
Jul 15 21:48:54.129349 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.129326    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-546bh\"/\"default-dockercfg-fkhhx\""
Jul 15 21:48:54.135621 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.135600    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"]
Jul 15 21:48:54.256416 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.256386    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-lib-modules\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.256575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.256443    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-proc\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.256575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.256517    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-sys\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.256575 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.256548    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-podres\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.256704 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.256583    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75xxl\" (UniqueName: \"kubernetes.io/projected/1a2615f0-887f-4124-9c95-af5d8a4c08af-kube-api-access-75xxl\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.357835 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.357800    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-lib-modules\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.357861    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-proc\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.357899    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-sys\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.357921    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-podres\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.357979    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75xxl\" (UniqueName: \"kubernetes.io/projected/1a2615f0-887f-4124-9c95-af5d8a4c08af-kube-api-access-75xxl\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.358010    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-lib-modules\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.358033    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-sys\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358056 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.358035    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-proc\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.358318 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.358092    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1a2615f0-887f-4124-9c95-af5d8a4c08af-podres\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.368522 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.368499    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75xxl\" (UniqueName: \"kubernetes.io/projected/1a2615f0-887f-4124-9c95-af5d8a4c08af-kube-api-access-75xxl\") pod \"perf-node-gather-daemonset-bmgjj\" (UID: \"1a2615f0-887f-4124-9c95-af5d8a4c08af\") " pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.436211 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.436179    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:54.564675 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:54.564650    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"]
Jul 15 21:48:54.566926 ip-10-0-128-44 kubenswrapper[2555]: W0715 21:48:54.566889    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1a2615f0_887f_4124_9c95_af5d8a4c08af.slice/crio-4fbdee16b7b3f247546c1c5ce9b65b1b2514f5548b71deb01cc4240c4b125ee2 WatchSource:0}: Error finding container 4fbdee16b7b3f247546c1c5ce9b65b1b2514f5548b71deb01cc4240c4b125ee2: Status 404 returned error can't find the container with id 4fbdee16b7b3f247546c1c5ce9b65b1b2514f5548b71deb01cc4240c4b125ee2
Jul 15 21:48:55.307588 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:55.307552    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj" event={"ID":"1a2615f0-887f-4124-9c95-af5d8a4c08af","Type":"ContainerStarted","Data":"e5f8cb36dd9750d036a78c961227904fe701bf103fe5bd4a04d5a968c8455604"}
Jul 15 21:48:55.308065 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:55.307609    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj" event={"ID":"1a2615f0-887f-4124-9c95-af5d8a4c08af","Type":"ContainerStarted","Data":"4fbdee16b7b3f247546c1c5ce9b65b1b2514f5548b71deb01cc4240c4b125ee2"}
Jul 15 21:48:55.308065 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:55.307661    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:48:55.325725 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:55.325678    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj" podStartSLOduration=1.325665307 podStartE2EDuration="1.325665307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:48:55.324179239 +0000 UTC m=+2539.656320355" watchObservedRunningTime="2026-07-15 21:48:55.325665307 +0000 UTC m=+2539.657806397"
Jul 15 21:48:56.957872 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:56.957841    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g5zkb_983aecbc-01b0-44df-b21e-edb44aeab3a2/dns/0.log"
Jul 15 21:48:56.998856 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:56.998830    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g5zkb_983aecbc-01b0-44df-b21e-edb44aeab3a2/kube-rbac-proxy/0.log"
Jul 15 21:48:57.177231 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:57.177202    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jsw54_200938e3-b583-4a3c-9bdb-fcbc25af1105/dns-node-resolver/0.log"
Jul 15 21:48:57.850937 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:57.850909    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qms62_16207249-d9af-4610-a473-fc018f680361/node-ca/0.log"
Jul 15 21:48:59.075147 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:59.075119    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bcb5978cd-w5cpd_d05f479c-075c-4e14-902f-6c3571e0418d/kube-auth-proxy/0.log"
Jul 15 21:48:59.204077 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:59.204043    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-svmk4_b9f5ec2c-610b-4946-98d4-429987b4a29d/payload-processing/0.log"
Jul 15 21:48:59.939041 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:48:59.939009    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b99lb_efca4669-f0d9-4457-9d83-558551a8c66f/serve-healthcheck-canary/0.log"
Jul 15 21:49:00.701159 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:00.701132    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2fr5_4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5/kube-rbac-proxy/0.log"
Jul 15 21:49:00.726709 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:00.726684    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2fr5_4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5/exporter/0.log"
Jul 15 21:49:00.753535 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:00.753508    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n2fr5_4c8880c8-d47f-4a2e-bcd1-0f5b9c63d5f5/extractor/0.log"
Jul 15 21:49:01.322553 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:01.322527    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-546bh/perf-node-gather-daemonset-bmgjj"
Jul 15 21:49:03.167987 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.167947    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6pscc_b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89/manager/1.log"
Jul 15 21:49:03.168342 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.167999    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6pscc_b8a8a0c6-a572-4aa5-bb46-cb0821bbbc89/manager/0.log"
Jul 15 21:49:03.227705 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.227606    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5s7kk_550cae76-7a34-4e8a-bad1-caca322ec031/manager/0.log"
Jul 15 21:49:03.296318 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.296237    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5s7kk_550cae76-7a34-4e8a-bad1-caca322ec031/manager/1.log"
Jul 15 21:49:03.417408 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.417367    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-prn5d_9564402b-7e1e-4275-aee5-adbe06c3c21b/server/0.log"
Jul 15 21:49:03.480012 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.479975    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j5z24_e87f8b62-a84a-479d-b8ea-44ec05e51993/manager/1.log"
Jul 15 21:49:03.482313 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.482294    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j5z24_e87f8b62-a84a-479d-b8ea-44ec05e51993/manager/0.log"
Jul 15 21:49:03.662042 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.662014    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-858d4f578f-rtk74_42978d06-f6bc-4e7b-9ca4-83606ade276f/manager/1.log"
Jul 15 21:49:03.683426 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.683402    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-858d4f578f-rtk74_42978d06-f6bc-4e7b-9ca4-83606ade276f/manager/0.log"
Jul 15 21:49:03.706941 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:03.706919    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-858d4f578f-rtk74_42978d06-f6bc-4e7b-9ca4-83606ade276f/copy-manifests/0.log"
Jul 15 21:49:04.604055 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:04.604031    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bcb5978cd-w5cpd_d05f479c-075c-4e14-902f-6c3571e0418d/kube-auth-proxy/0.log"
Jul 15 21:49:04.702659 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:04.702625    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-svmk4_b9f5ec2c-610b-4946-98d4-429987b4a29d/payload-processing/0.log"
Jul 15 21:49:06.119341 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:06.119257    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bcb5978cd-w5cpd_d05f479c-075c-4e14-902f-6c3571e0418d/kube-auth-proxy/0.log"
Jul 15 21:49:06.220563 ip-10-0-128-44 kubenswrapper[2555]: I0715 21:49:06.220527    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-svmk4_b9f5ec2c-610b-4946-98d4-429987b4a29d/payload-processing/0.log"